-- Logs begin at Thu 2022-09-01 15:49:54 UTC, end at Thu 2022-09-01 16:56:43 UTC. -- Sep 01 16:21:59.493808 dv-cinder-852210-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 16:21:59.501114 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 16:21:59.503115 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 16:21:59 2022] *** Sep 01 16:21:59.503115 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 16:21:59.503115 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 16:21:59.503115 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: nodename: dv-cinder-852210-9-iscsi Sep 01 16:21:59.503115 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: machine: x86_64 Sep 01 16:21:59.503115 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: clock source: unix Sep 01 16:21:59.503115 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: pcre jit disabled Sep 01 16:21:59.503115 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: detected number of CPU cores: 4 Sep 01 16:21:59.503115 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: current working directory: / Sep 01 16:21:59.503115 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: detected binary path: /usr/bin/uwsgi-core Sep 01 16:21:59.503115 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: your processes number limit is 47680 Sep 01 16:21:59.503115 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: your memory page size is 4096 bytes Sep 01 16:21:59.503115 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: detected max file descriptor number: 2048 Sep 01 16:21:59.503115 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: lock engine: pthread robust mutexes Sep 01 16:21:59.504089 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: thunder lock: enabled Sep 01 16:21:59.504089 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 16:21:59.504089 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 16:21:59.543675 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: Python main interpreter initialized at 0x55b3381029c0 Sep 01 16:21:59.543675 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: python threads support enabled Sep 01 16:21:59.543675 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: your server socket listen backlog is limited to 100 connections Sep 01 16:21:59.543675 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: your mercy for graceful operations on workers is 90 seconds Sep 01 16:21:59.544158 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 16:21:59.544239 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: *** Operational MODE: preforking *** Sep 01 16:21:59.544537 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: *** uWSGI is running in multiple interpreter mode *** Sep 01 16:21:59.544537 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: spawned uWSGI master process (pid: 81508) Sep 01 16:21:59.544745 dv-cinder-852210-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 16:21:59.547676 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: spawned uWSGI worker 1 (pid: 81522, cores: 1) Sep 01 16:21:59.547676 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: spawned uWSGI worker 2 (pid: 81523, cores: 1) Sep 01 16:21:59.547676 dv-cinder-852210-9-iscsi devstack@keystone.service[81508]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 16:22:03.791499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:22:03.791499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: from cryptography.utils import int_from_bytes Sep 01 16:22:03.792277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:22:03.792277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: from cryptography.utils import int_from_bytes Sep 01 16:22:03.793298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:22:03.793298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: from cryptography.utils import int_from_bytes Sep 01 16:22:03.793783 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:22:03.793783 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: from cryptography.utils import int_from_bytes Sep 01 16:22:03.841456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 16:22:03.841665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 16:22:03.841822 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] command line args: [] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 16:22:03.842084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 16:22:03.842212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ================================================================================ {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 16:22:03.842490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] admin_token = **** {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.842733 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] config_dir = [] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.842863 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.843073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] config_source = [] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.843202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] debug = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.843435 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.843698 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 16:22:03.843698 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 16:22:03.843829 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.843915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] command line args: [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 16:22:03.843994 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.844064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 16:22:03.844224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.844301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ================================================================================ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 16:22:03.844466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.844589 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] admin_token = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.844661 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] list_limit = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.844733 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] config_dir = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.844859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] log_config_append = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.844941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.845002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.845057 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] config_source = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.845164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] log_dir = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.845244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] debug = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.845310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] log_file = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.845412 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.845537 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.845632 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.845805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.845930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.846004 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.846073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.846270 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.846398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.846478 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.846560 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.846625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.846697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_config_append = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.846827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.846965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.846965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_dir = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.847119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.847194 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_file = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.847261 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.847335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.847490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.847597 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.847733 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.847918 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.848016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.848103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.848178 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.848244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.848397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.848479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.848644 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.848767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.848865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.849113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.849230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.849304 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] publish_errors = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.849423 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.849497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.849607 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.849728 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.849807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.849916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.849996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.850337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.850431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.850547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.850638 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.850712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] standard_threads = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.851054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.851171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.851256 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] publish_errors = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.851345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.851425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.851500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.851599 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] use_journal = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.851685 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.851763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] use_json = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.851859 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.851995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] use_stderr = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.852109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] use_syslog = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.852185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.852326 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.852473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.852555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.852633 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] standard_threads = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.852798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.852879 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.853039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.853126 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.853196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.853264 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.853366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_journal = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.853502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.853581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_json = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.853711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_stderr = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.853791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.853859 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_syslog = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.853961 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.854087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:22:03.854190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.854344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.854431 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.854585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.854664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.854842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.854921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.855080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.855167 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.855326 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.855412 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.855563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.855641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.855800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.855879 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.856064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.856156 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.856289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.856369 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.856538 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.856620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.856786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.856866 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.857018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.857095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.857246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.857328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.857483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.857561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.857710 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.857787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.857939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.858016 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.858178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.858263 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.858398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.858482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.858621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.858702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.858844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.858921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.859071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.859156 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.859311 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.859386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.859550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.859639 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.859773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.859858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.860024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.860101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.860266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.860415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.860495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.860577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.860724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.860805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.860965 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.861046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.861190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.861277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.861417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.861498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.861644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.861718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.861866 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.861949 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.862091 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.862204 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.862312 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.862388 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.862535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.862622 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.862767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.862843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.863006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.863079 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.863228 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.863385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.863459 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.863533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.863679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.863757 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.863930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.864043 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.864164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.864249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.864397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.864473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.864634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.864718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.864879 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.864956 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.865110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.865186 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.865338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.865420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.865565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.865664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.865832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.865989 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.866090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.866191 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.866285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.866377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.866511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.866624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.866741 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.866822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.866927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.867012 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.867180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.867258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.867411 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.867498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.867639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.867715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.867865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.868000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.868119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.868234 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.868365 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.868442 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.868604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.868689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.868834 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.868964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.869075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.869193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.869307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.869386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.869533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.869608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.869762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.869876 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.869994 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.870078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.870224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.870300 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.870454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.870534 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.870692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.870806 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.870925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.871002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.871165 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.871243 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.871397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.871471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.871624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.871699 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.871854 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.871984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.872106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.872339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.872449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.872591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.872680 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.872826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.872924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.873067 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.873163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.873300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.873413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.873532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.873651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.873771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.873888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.874003 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.874124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.874233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.874364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.874462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.874598 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.874674 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.874856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.874936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.875101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.875177 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.875327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.875411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.875551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.875636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.875785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.875861 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.876039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.876116 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.876277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.876365 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.876507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.876582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.876741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.876815 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.876973 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.877061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.877206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.877289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.877428 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] auth.external = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.877503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.877655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.877738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.877882 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.877963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.878124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.878202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.878364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.878474 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] auth.password = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.878584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] auth.token = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.878668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.878817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.878895 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.879047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.879123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.879287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.879448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.879527 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.879702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.879779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.879972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.external = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.880052 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.880201 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.880278 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.880431 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] credential.caching = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.880509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.880675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.880751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.880916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.880995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.password = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.881156 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.881234 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.token = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.881431 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.881514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.881701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.881780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.881933 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.882043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.882174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.882423 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.882514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.882656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.882734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.882902 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.882979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.883157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.883241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.883400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.883478 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.883655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.883733 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.883936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.884029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.884161 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.884239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.884393 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] federation.caching = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.884480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.884625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.884711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.884864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.884940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.885101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.885176 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.885325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.885409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.885565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.885643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.885796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.885873 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.886042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.886127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.886271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.886348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.886522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.886599 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.886755 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.886834 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.886999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.887076 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.887230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity.caching = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.887306 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.887472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.887550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.887714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.887791 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.887967 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.888048 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.888201 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.888280 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.888448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.888525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.888673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.888749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.888899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.888975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.889138 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.889216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.889384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.889463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.889610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.889689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.889836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.889914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.890060 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.890137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.890299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.890377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.890539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.890615 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.890778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.890855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.891013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.891090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.891254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.891329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.891484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.891559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.891724 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.891802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.891980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.892059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.892208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.892284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.892440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.892522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.892664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.892741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.892889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.892964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.893122 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.893200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.893358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.893435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.893603 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.893678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.893826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.893903 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.894069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.894146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.894311 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.894388 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.894543 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.894621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.894787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.894863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.895014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.895089 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.895239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.895316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.895469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.895544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.895690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.895765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.895952 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.896034 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.896188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.896275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.896424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.896510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.896651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.896728 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.896880 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.896955 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.897148 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.897238 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.897392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.897469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.897618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.897779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.897857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.898030 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.898109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.898264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.898341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.898507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.898587 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.898734 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.898811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.898966 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.899043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.899194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.899271 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.899427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.899506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.899662 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.899738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.899934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.900014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.900156 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.900235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.900400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.900478 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.900630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.900705 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.900858 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.900935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.901083 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.901166 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.901313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.901390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.901541 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.901620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.901773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.901850 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.901998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.902075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.902227 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.902305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.902474 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.902595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.902712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.902808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.902946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.903023 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.903182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.903258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.903408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.903485 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.903656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.903734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.903895 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.904016 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.904137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.904213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.904364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.904441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.904594 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.904672 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.904828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.904903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.905061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.905139 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.905290 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.905365 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.905539 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.905618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.905760 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.905837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.905999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.906076 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.906230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.906306 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.906464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.906550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.906689 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.906771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.906928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.907032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.907162 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.907273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.907397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.907483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.907630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.907708 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.907878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.908018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] resource.caching = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.908152 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.908229 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.908392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.908469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.908648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.908727 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.908880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.908963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.909123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.909199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.909357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.909434 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.909590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.909665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.909820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.909896 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.910054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.910132 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.910286 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] role.caching = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.910363 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.910515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.910590 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] role.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.910742 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.910818 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.910985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.911062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.911227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.911383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.911458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.911529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.911694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.911769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.911965 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.912057 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.912210 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.912287 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.912446 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.912561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.912754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.912884 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.912988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.913170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.913248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.913463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.913588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.913720 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.913844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] role.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.913967 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.914090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] role.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.914211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.914345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.914475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.914604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.914727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.914850 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.914972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.915094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.915224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.915347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.915468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.915590 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.915708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.915831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.916000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.916122 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.916245 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.916367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.916488 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.916609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.916738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.916861 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.916983 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.917105 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.917227 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.917371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.917500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.917632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.917632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.917863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.917985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.918113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.918236 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.918357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.918479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.918600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.918722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.918850 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.918972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] token.caching = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.919093 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.919230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.919354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.919476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.919603 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.919726 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.919847 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.919998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.920116 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.920234 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.920361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.920480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.920597 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.920716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.920833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.920967 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.921093 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.921212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.921212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.921434 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.921552 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.921670 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.921795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.921913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.922030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.922147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.922264 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.922381 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.922506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.922665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.922798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.922918 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.922918 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.923158 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.923274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.923399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.connection = **** {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.923519 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.923638 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.923755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.923873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.923873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.924155 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.924287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.924410 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.924532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.924654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.924776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.924776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.925013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.925135 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.925256 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.925377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.925497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.925627 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.925750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.925871 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.926002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.926124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.926245 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.926374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.926496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.926627 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.926757 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.926879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.927000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.927095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.927214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.927340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.927461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.927582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.927582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 16:22:03.927822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.927822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.927822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.927822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.927822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.928429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.928429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.928671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.928916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.929163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.929397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.929646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.929884 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.930129 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.930361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.930601 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.930844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:22:03.931017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 16:22:03.956302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:03.956526 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:03.956743 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:03.959749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:03.959929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:03.960077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:03.965310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:03.968819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:03.993140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:03.993353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:03.994862 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:03.995101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.046305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.047735 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.094812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.095018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.096784 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.097025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.253820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.254055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.254220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.254462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.254649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.254892 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.255024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.255164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.255365 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.255562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.255751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.255960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.277730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.277940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.278180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.278397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.278520 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.278656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.278736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.278798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.278929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.279008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.279123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.279330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.279546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.279758 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.290091 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.291149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.302953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.303085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.303328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.303625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.303835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.304065 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.304195 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.304330 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.304429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.304719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.304935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.305150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.305364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:22:04.321103 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.321310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:22:04.321649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.321844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:22:04.321967 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.322124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.322207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:22:04.322354 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.322515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.322608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.322728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:22:04.322817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.323274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.323545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.323813 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.324113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.332760 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.333040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.333307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.333582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.334102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.334386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.334711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.335125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.340933 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.342304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.343812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.344118 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.344403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.344682 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.344960 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.345214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.345356 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.345494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.345575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.345691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.345768 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.345987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.346114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.346249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.346329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.346503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.346641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.346930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.347246 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.347520 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.347784 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.348151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.369423 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.370883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.371113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.371251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.372605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.372894 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.375673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.375981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.376264 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.377596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.377885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.378166 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.382796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.383087 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.384667 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.384956 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.387289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.387615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.387917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.388224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.388494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.389086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.389382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.389672 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.389959 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.390235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.390477 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.390603 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.390742 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.390828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.391021 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.391104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.406686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.406985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.408602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.408880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.410843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.413034 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.414077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.414373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.416208 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.416511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.418712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.420789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.421858 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.424094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.424978 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.425278 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.425562 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.425843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.426119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.426546 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.427236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.427551 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.427841 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.428160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.428449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.428735 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.442430 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.442718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.442987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.443250 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.445118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.445407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.445676 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.445957 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.450030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.451660 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.453220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.453575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.453852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.454805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.456370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.456494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.456628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.456831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.457070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.457343 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.457639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.459418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.459745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.460266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.460575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.460868 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.469144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.469442 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.469725 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.469998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.470271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.470539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.470817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.472270 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.472551 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.472836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.473120 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.473405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.473683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.473970 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.486441 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.489503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.489895 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.492586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.492901 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.494026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.495493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.495775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.495915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.496053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.497413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.498917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.499207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.499500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.502689 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.504188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.505979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.507122 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.507412 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.507687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.507921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.508008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.510759 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.511052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.511349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.511649 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.515917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.516214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.516492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.519681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.519996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.520288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.523398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.523681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.523992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.524265 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.524529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.524795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.525041 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.525352 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.525624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.525896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.526160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.526400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.526658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.526926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.527487 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:22:04.527774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.528102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.528385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.528665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.528943 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.529233 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.529508 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.529897 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.530433 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.530717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.530986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.531267 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.531555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:22:04.560888 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b3381029c0 pid: 81523 (default app) Sep 01 16:22:04.564305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc40d9d-27eb-4eec-9044-74c28403daf7 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:04.564562 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc40d9d-27eb-4eec-9044-74c28403daf7 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:04.564852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc40d9d-27eb-4eec-9044-74c28403daf7 None None] PATH_INFO: `/v3` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:04.565784 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b3381029c0 pid: 81522 (default app) Sep 01 16:22:04.569123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1/1] 10.222.0.62 () {50 vars in 791 bytes} [Thu Sep 1 16:22:04 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:11.326398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fd383-328c-448e-808f-aeba58f53966 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:11.327224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fd383-328c-448e-808f-aeba58f53966 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:11.327224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fd383-328c-448e-808f-aeba58f53966 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:11.337854 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1/2] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:11 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:11.349426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60f25d-e6b6-416c-a32c-7330e77d2acf None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:11.350214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60f25d-e6b6-416c-a32c-7330e77d2acf None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:11.350445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60f25d-e6b6-416c-a32c-7330e77d2acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:11.419920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: INFO dbcounter [None req-df60f25d-e6b6-416c-a32c-7330e77d2acf None None] Registered counter for database keystone Sep 01 16:22:11.441023 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG oslo_db.sqlalchemy.engines [None req-df60f25d-e6b6-416c-a32c-7330e77d2acf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81523) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 16:22:11.450537 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writer thread running {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 16:22:11.643611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG passlib.handlers.bcrypt [None req-df60f25d-e6b6-416c-a32c-7330e77d2acf None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81523) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 16:22:11.643930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG passlib.handlers.bcrypt [None req-df60f25d-e6b6-416c-a32c-7330e77d2acf None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81523) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 16:22:11.702669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-df60f25d-e6b6-416c-a32c-7330e77d2acf None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:11.704633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-df60f25d-e6b6-416c-a32c-7330e77d2acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:11.848940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 2/3] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 501 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 16:22:12.369788 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fd383-328c-448e-808f-aeba58f53966 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:12.452073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: INFO dbcounter [None req-3e8fd383-328c-448e-808f-aeba58f53966 None None] Registered counter for database keystone Sep 01 16:22:12.477463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG oslo_db.sqlalchemy.engines [None req-3e8fd383-328c-448e-808f-aeba58f53966 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81522) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 16:22:12.489047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writer thread running {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 16:22:12.608620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fd383-328c-448e-808f-aeba58f53966 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:12.614085 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d5ab5-d08e-438f-aa77-7a3aed9a8f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:12.614704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d5ab5-d08e-438f-aa77-7a3aed9a8f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:12.614971 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d5ab5-d08e-438f-aa77-7a3aed9a8f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:12.615237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d5ab5-d08e-438f-aa77-7a3aed9a8f2b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:12.624356 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d5ab5-d08e-438f-aa77-7a3aed9a8f2b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:12.675631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d69d5ab5-d08e-438f-aa77-7a3aed9a8f2b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 16:22:12.675953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d5ab5-d08e-438f-aa77-7a3aed9a8f2b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:12.686300 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 2/4] 10.222.0.62 () {58 vars in 1324 bytes} [Thu Sep 1 16:22:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:22:12.693403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60f25d-e6b6-416c-a32c-7330e77d2acf None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:12.709942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60f25d-e6b6-416c-a32c-7330e77d2acf None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:12.716724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61825c1d-9aca-481d-a944-66996448ce2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:12.718492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61825c1d-9aca-481d-a944-66996448ce2d admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:12.718954 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61825c1d-9aca-481d-a944-66996448ce2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:12.719356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61825c1d-9aca-481d-a944-66996448ce2d admin admin] PATH_INFO: `/v3/projects/026b4ed9c3cf48a48d86818cba23c13c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:12.722238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61825c1d-9aca-481d-a944-66996448ce2d admin admin] RBAC: Authorizing `identity:get_project(project_id=026b4ed9c3cf48a48d86818cba23c13c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:12.777884 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: WARNING keystone.common.rbac_enforcer.enforcer [None req-61825c1d-9aca-481d-a944-66996448ce2d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 16:22:12.778231 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61825c1d-9aca-481d-a944-66996448ce2d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:12.782966 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 3/5] 10.222.0.62 () {56 vars in 1207 bytes} [Thu Sep 1 16:22:12 2022] GET /identity/v3/projects/026b4ed9c3cf48a48d86818cba23c13c => generated 344 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:14.640957 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca8c17-3fd0-4336-9b6f-00795e9a378d None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:14.641854 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca8c17-3fd0-4336-9b6f-00795e9a378d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:14.641854 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca8c17-3fd0-4336-9b6f-00795e9a378d None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:14.648639 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 3/6] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:14.654812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39af8fb-868d-45c4-8fb4-ba5ec4e7d145 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:14.655255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39af8fb-868d-45c4-8fb4-ba5ec4e7d145 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:14.655412 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39af8fb-868d-45c4-8fb4-ba5ec4e7d145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:14.679720 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-a39af8fb-868d-45c4-8fb4-ba5ec4e7d145 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:14.681346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a39af8fb-868d-45c4-8fb4-ba5ec4e7d145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:14.691623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 4/7] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 16:22:14.965121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca8c17-3fd0-4336-9b6f-00795e9a378d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:14.983752 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca8c17-3fd0-4336-9b6f-00795e9a378d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:14.990025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e8e27-36a1-46f3-8a61-761d0a0f2cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:14.990639 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e8e27-36a1-46f3-8a61-761d0a0f2cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:14.990911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e8e27-36a1-46f3-8a61-761d0a0f2cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:14.991180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e8e27-36a1-46f3-8a61-761d0a0f2cb9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:14.999183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e8e27-36a1-46f3-8a61-761d0a0f2cb9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:15.000998 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e8e27-36a1-46f3-8a61-761d0a0f2cb9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:15.009521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 4/8] 10.222.0.62 () {58 vars in 1324 bytes} [Thu Sep 1 16:22:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:22:15.015593 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39af8fb-868d-45c4-8fb4-ba5ec4e7d145 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:15.025244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39af8fb-868d-45c4-8fb4-ba5ec4e7d145 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:15.030853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1399039-afd3-44e3-9485-726daf288cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:15.031493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1399039-afd3-44e3-9485-726daf288cdf admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:15.031771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1399039-afd3-44e3-9485-726daf288cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:15.032068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1399039-afd3-44e3-9485-726daf288cdf admin admin] PATH_INFO: `/v3/users/82b69c02f3fd4ddc81686f03143f0651` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:15.033509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1399039-afd3-44e3-9485-726daf288cdf admin admin] RBAC: Authorizing `identity:get_user(user_id=82b69c02f3fd4ddc81686f03143f0651)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:15.035360 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1399039-afd3-44e3-9485-726daf288cdf admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:15.036940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 5/9] 10.222.0.62 () {56 vars in 1198 bytes} [Thu Sep 1 16:22:15 2022] GET /identity/v3/users/82b69c02f3fd4ddc81686f03143f0651 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:17.012516 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bf6f5-2648-454b-8d0c-a117b6cacfe5 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:17.013036 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bf6f5-2648-454b-8d0c-a117b6cacfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:17.013036 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bf6f5-2648-454b-8d0c-a117b6cacfe5 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:17.014310 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 5/10] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:17.019605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578816a8-5283-461c-9a49-5adf2c4ae7d2 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:17.020078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578816a8-5283-461c-9a49-5adf2c4ae7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:17.020173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578816a8-5283-461c-9a49-5adf2c4ae7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:17.030002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bd5c3-ee50-4f9b-91f4-840428d9c9bf None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:17.030285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bd5c3-ee50-4f9b-91f4-840428d9c9bf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:17.030547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bd5c3-ee50-4f9b-91f4-840428d9c9bf None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:17.031822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 6/11] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:17.036990 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d40e5-b09a-4198-b681-9d7d33a8f65e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:17.037269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d40e5-b09a-4198-b681-9d7d33a8f65e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:17.037493 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d40e5-b09a-4198-b681-9d7d33a8f65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:17.039618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-578816a8-5283-461c-9a49-5adf2c4ae7d2 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:17.040827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-578816a8-5283-461c-9a49-5adf2c4ae7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:17.080574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG passlib.handlers.bcrypt [None req-c81d40e5-b09a-4198-b681-9d7d33a8f65e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81522) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 16:22:17.081022 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG passlib.handlers.bcrypt [None req-c81d40e5-b09a-4198-b681-9d7d33a8f65e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81522) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 16:22:17.126893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 6/12] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:17.129368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c81d40e5-b09a-4198-b681-9d7d33a8f65e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:17.130953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c81d40e5-b09a-4198-b681-9d7d33a8f65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:17.232016 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 7/13] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:17.392518 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578816a8-5283-461c-9a49-5adf2c4ae7d2 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:17.418756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:17.503186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d40e5-b09a-4198-b681-9d7d33a8f65e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:17.503955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578816a8-5283-461c-9a49-5adf2c4ae7d2 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:17.540538 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7c727-b6b8-437f-9ffd-b2b17182564e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:17.541180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7c727-b6b8-437f-9ffd-b2b17182564e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:17.541468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7c727-b6b8-437f-9ffd-b2b17182564e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:17.541737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7c727-b6b8-437f-9ffd-b2b17182564e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:17.552642 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7c727-b6b8-437f-9ffd-b2b17182564e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-a0a7c727-b6b8-437f-9ffd-b2b17182564e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:17.561748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:17.562765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:17.563740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:17.563740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:17.563740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:17.563740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:17.563740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:17.563740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:17.563740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:17.563740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:17.563740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:17.563740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:17.563740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:17.563740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:17.563740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 7/14] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:17.568657 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7c727-b6b8-437f-9ffd-b2b17182564e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:17.605099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d40e5-b09a-4198-b681-9d7d33a8f65e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:17.607384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:17.644628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4b57c-eff2-4079-bc99-29b946555acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:17.645422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4b57c-eff2-4079-bc99-29b946555acf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:17.645707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4b57c-eff2-4079-bc99-29b946555acf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:17.645978 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4b57c-eff2-4079-bc99-29b946555acf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:17.651304 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:17.670623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7c727-b6b8-437f-9ffd-b2b17182564e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:17.703328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd3bc1-3019-4182-ba74-aad21c903e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:17.704306 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd3bc1-3019-4182-ba74-aad21c903e31 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:17.704576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd3bc1-3019-4182-ba74-aad21c903e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:17.704843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd3bc1-3019-4182-ba74-aad21c903e31 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:17.705573 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd3bc1-3019-4182-ba74-aad21c903e31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:17.705872 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd3bc1-3019-4182-ba74-aad21c903e31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:17.707275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd3bc1-3019-4182-ba74-aad21c903e31 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:17.714053 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4b57c-eff2-4079-bc99-29b946555acf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:17.715608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4b57c-eff2-4079-bc99-29b946555acf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:17.719064 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 8/15] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:17.724930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd3bc1-3019-4182-ba74-aad21c903e31 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:17.819197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 8/16] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:17.825077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4b57c-eff2-4079-bc99-29b946555acf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:17.827934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:17.832522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd3bc1-3019-4182-ba74-aad21c903e31 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:17.865908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10cda3-86ff-4acb-9e73-19a8210f2f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:17.866540 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10cda3-86ff-4acb-9e73-19a8210f2f2d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:17.866815 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10cda3-86ff-4acb-9e73-19a8210f2f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:17.867227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10cda3-86ff-4acb-9e73-19a8210f2f2d None admin] PATH_INFO: `/v3/users/82b69c02f3fd4ddc81686f03143f0651` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:17.868391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10cda3-86ff-4acb-9e73-19a8210f2f2d None admin] RBAC: Authorizing `identity:get_user(user_id=82b69c02f3fd4ddc81686f03143f0651)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:17.869905 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10cda3-86ff-4acb-9e73-19a8210f2f2d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:17.871466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 9/17] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:17 2022] GET /identity/v3/users/82b69c02f3fd4ddc81686f03143f0651 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:17.877124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10cda3-86ff-4acb-9e73-19a8210f2f2d None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:17.891344 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:17.925442 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4b57c-eff2-4079-bc99-29b946555acf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:17.959623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19bdb6-ba81-4957-887e-82974b7151e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:17.960494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19bdb6-ba81-4957-887e-82974b7151e5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:17.960827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19bdb6-ba81-4957-887e-82974b7151e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:17.961112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19bdb6-ba81-4957-887e-82974b7151e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:17.975021 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10cda3-86ff-4acb-9e73-19a8210f2f2d None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:17.979432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19bdb6-ba81-4957-887e-82974b7151e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:17.980962 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19bdb6-ba81-4957-887e-82974b7151e5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-db19bdb6-ba81-4957-887e-82974b7151e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:22:17.999291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:18.000257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:22:18.001037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:22:18.001037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:18.001037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:18.001037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 9/18] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:18.006342 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19bdb6-ba81-4957-887e-82974b7151e5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:18.009964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ccdae-8b31-43ef-b6ef-b199bfd83010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:18.010598 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ccdae-8b31-43ef-b6ef-b199bfd83010 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:18.010870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ccdae-8b31-43ef-b6ef-b199bfd83010 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:18.011135 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ccdae-8b31-43ef-b6ef-b199bfd83010 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:18.012288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ccdae-8b31-43ef-b6ef-b199bfd83010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:18.013748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ccdae-8b31-43ef-b6ef-b199bfd83010 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:18.015305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 10/19] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:18.021237 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ccdae-8b31-43ef-b6ef-b199bfd83010 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:18.067986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:18.114776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:18.121024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19bdb6-ba81-4957-887e-82974b7151e5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:18.135697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ccdae-8b31-43ef-b6ef-b199bfd83010 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:18.156033 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdcf9a-1dae-469a-ba96-68eb6222eddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:18.156647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdcf9a-1dae-469a-ba96-68eb6222eddc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:18.156924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdcf9a-1dae-469a-ba96-68eb6222eddc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:18.157187 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdcf9a-1dae-469a-ba96-68eb6222eddc None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:18.157830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdcf9a-1dae-469a-ba96-68eb6222eddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:18.158126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdcf9a-1dae-469a-ba96-68eb6222eddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:18.159527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdcf9a-1dae-469a-ba96-68eb6222eddc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:18.171271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5dbdc-4b17-4fe0-bea4-8f1e941fcf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:18.172064 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5dbdc-4b17-4fe0-bea4-8f1e941fcf9a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:18.172383 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5dbdc-4b17-4fe0-bea4-8f1e941fcf9a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:18.172778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5dbdc-4b17-4fe0-bea4-8f1e941fcf9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:18.173564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5dbdc-4b17-4fe0-bea4-8f1e941fcf9a None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.domain.id=default, user.id=82b69c02f3fd4ddc81686f03143f0651) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:18.173865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5dbdc-4b17-4fe0-bea4-8f1e941fcf9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:18.175371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5dbdc-4b17-4fe0-bea4-8f1e941fcf9a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:18.177099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 10/20] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:18.182628 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 11/21] 10.222.0.62 () {56 vars in 1322 bytes} [Thu Sep 1 16:22:18 2022] GET /identity/v3/role_assignments?user.id=82b69c02f3fd4ddc81686f03143f0651&scope.domain.id=default&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 233 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:20.252097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3706f-a2e8-4f4f-940c-6942be4abf31 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.252630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3706f-a2e8-4f4f-940c-6942be4abf31 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.252630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3706f-a2e8-4f4f-940c-6942be4abf31 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.253983 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 11/22] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:20.259497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9e26b-8c00-4434-b71d-0e1c3f290fee None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.259930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9e26b-8c00-4434-b71d-0e1c3f290fee None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.260123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9e26b-8c00-4434-b71d-0e1c3f290fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.284893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-43b9e26b-8c00-4434-b71d-0e1c3f290fee None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:20.286106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-43b9e26b-8c00-4434-b71d-0e1c3f290fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:20.378068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 12/23] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:22:20.501727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a319a6-488c-4933-a7c5-f8f24e9165ee None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.502015 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a319a6-488c-4933-a7c5-f8f24e9165ee None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.502317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a319a6-488c-4933-a7c5-f8f24e9165ee None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.503612 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 12/24] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:20.509140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952caf1-5b57-4c5f-9da3-a2923145a412 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.509424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952caf1-5b57-4c5f-9da3-a2923145a412 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.509697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952caf1-5b57-4c5f-9da3-a2923145a412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.536916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e952caf1-5b57-4c5f-9da3-a2923145a412 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:20.538115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e952caf1-5b57-4c5f-9da3-a2923145a412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:20.583793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca86ec-ab1c-4243-b203-5de393fdb411 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.584154 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca86ec-ab1c-4243-b203-5de393fdb411 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.584369 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca86ec-ab1c-4243-b203-5de393fdb411 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.585706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 13/25] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:22:20.593204 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54692a7a-43e1-4053-b7ef-ef09a94debba None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.593489 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54692a7a-43e1-4053-b7ef-ef09a94debba None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.593766 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54692a7a-43e1-4053-b7ef-ef09a94debba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.623119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-54692a7a-43e1-4053-b7ef-ef09a94debba None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:20.624360 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-54692a7a-43e1-4053-b7ef-ef09a94debba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:20.665118 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 13/26] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:20.723383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 14/27] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:20.762981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952caf1-5b57-4c5f-9da3-a2923145a412 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:20.769619 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:20.914045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952caf1-5b57-4c5f-9da3-a2923145a412 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:20.956473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e09f0b-2a5d-497a-8588-a836015e73f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:20.957130 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e09f0b-2a5d-497a-8588-a836015e73f1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:20.957406 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e09f0b-2a5d-497a-8588-a836015e73f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:20.957673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e09f0b-2a5d-497a-8588-a836015e73f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:20.975862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e09f0b-2a5d-497a-8588-a836015e73f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-97e09f0b-2a5d-497a-8588-a836015e73f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:20.980983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:20.981796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:20.982522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:20.982522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:20.982522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:20.982522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:20.982522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:20.982522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:20.982522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:20.982522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:20.982522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:20.982522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:20.982522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:20.982522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:20.982522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 14/28] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:20.988565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54692a7a-43e1-4053-b7ef-ef09a94debba None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:21.015310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e09f0b-2a5d-497a-8588-a836015e73f1 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:21.073065 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:21.114592 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:21.161781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e09f0b-2a5d-497a-8588-a836015e73f1 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:21.201208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774b7ed-daf6-4d68-94f8-b942f6eafe7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:21.201830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774b7ed-daf6-4d68-94f8-b942f6eafe7e None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.202111 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774b7ed-daf6-4d68-94f8-b942f6eafe7e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.202377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774b7ed-daf6-4d68-94f8-b942f6eafe7e None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.203066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774b7ed-daf6-4d68-94f8-b942f6eafe7e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:21.205292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774b7ed-daf6-4d68-94f8-b942f6eafe7e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:21.210131 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54692a7a-43e1-4053-b7ef-ef09a94debba None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:21.272102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde3556-7def-493c-ae27-b700d41e602f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:21.273614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde3556-7def-493c-ae27-b700d41e602f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.274121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde3556-7def-493c-ae27-b700d41e602f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.274749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde3556-7def-493c-ae27-b700d41e602f None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.275620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde3556-7def-493c-ae27-b700d41e602f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:21.276092 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde3556-7def-493c-ae27-b700d41e602f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:21.277778 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde3556-7def-493c-ae27-b700d41e602f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:21.292055 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 15/29] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:21.293209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde3556-7def-493c-ae27-b700d41e602f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:21.359919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 15/30] 10.222.0.62 () {60 vars in 1131 bytes} [Thu Sep 1 16:22:21 2022] POST /identity/v3/roles => generated 219 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:21.362112 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd515966-e454-4ce8-a904-f424228e1a35 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.362541 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd515966-e454-4ce8-a904-f424228e1a35 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.362972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd515966-e454-4ce8-a904-f424228e1a35 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.364730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 16/31] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:21.365817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd515966-e454-4ce8-a904-f424228e1a35 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:21.414792 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde3556-7def-493c-ae27-b700d41e602f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:21.428298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:21.493593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4647467-68ff-4aeb-95ba-1319048b58a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:21.494747 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4647467-68ff-4aeb-95ba-1319048b58a6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.496565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4647467-68ff-4aeb-95ba-1319048b58a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.497033 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4647467-68ff-4aeb-95ba-1319048b58a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.520161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4647467-68ff-4aeb-95ba-1319048b58a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:21.523156 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4647467-68ff-4aeb-95ba-1319048b58a6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c4647467-68ff-4aeb-95ba-1319048b58a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:22:21.546029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:21.546936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:22:21.547746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:22:21.547746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:22:21.547746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:21.550109 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 16/32] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:21.550799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd515966-e454-4ce8-a904-f424228e1a35 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:21.551731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e898c62-3243-49ef-8f70-dc83671ac48c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.552237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e898c62-3243-49ef-8f70-dc83671ac48c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.552791 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e898c62-3243-49ef-8f70-dc83671ac48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.577580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5e898c62-3243-49ef-8f70-dc83671ac48c None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:21.579483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5e898c62-3243-49ef-8f70-dc83671ac48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:21.591248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:21.603168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f5651-410f-49f6-86eb-3c7551cd9102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:21.603872 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f5651-410f-49f6-86eb-3c7551cd9102 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.604396 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f5651-410f-49f6-86eb-3c7551cd9102 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.604569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f5651-410f-49f6-86eb-3c7551cd9102 None admin] PATH_INFO: `/v3/users/82b69c02f3fd4ddc81686f03143f0651` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.605971 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f5651-410f-49f6-86eb-3c7551cd9102 None admin] RBAC: Authorizing `identity:get_user(user_id=82b69c02f3fd4ddc81686f03143f0651)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:21.608212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f5651-410f-49f6-86eb-3c7551cd9102 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:21.609826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 17/33] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:21 2022] GET /identity/v3/users/82b69c02f3fd4ddc81686f03143f0651 => generated 247 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:21.610713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f5651-410f-49f6-86eb-3c7551cd9102 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:21.660789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 17/34] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:21.664242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e898c62-3243-49ef-8f70-dc83671ac48c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:21.753352 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f5651-410f-49f6-86eb-3c7551cd9102 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:21.780849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:21.791557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e898c62-3243-49ef-8f70-dc83671ac48c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:21.799217 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab977b1d-a254-45ba-9376-8f2c9c7ef4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:21.800400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab977b1d-a254-45ba-9376-8f2c9c7ef4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.800686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab977b1d-a254-45ba-9376-8f2c9c7ef4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.800960 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab977b1d-a254-45ba-9376-8f2c9c7ef4e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.801618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab977b1d-a254-45ba-9376-8f2c9c7ef4e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:21.801922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab977b1d-a254-45ba-9376-8f2c9c7ef4e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:21.804227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab977b1d-a254-45ba-9376-8f2c9c7ef4e7 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:21.823196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 18/35] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:21.829201 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab977b1d-a254-45ba-9376-8f2c9c7ef4e7 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:21.874485 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddf618-d2a3-43ad-a53b-300fbccf7188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:21.875153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddf618-d2a3-43ad-a53b-300fbccf7188 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.875432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddf618-d2a3-43ad-a53b-300fbccf7188 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.875706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddf618-d2a3-43ad-a53b-300fbccf7188 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.877118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddf618-d2a3-43ad-a53b-300fbccf7188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:21.884163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddf618-d2a3-43ad-a53b-300fbccf7188 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:21.886001 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 18/36] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:21.887406 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:21.892020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddf618-d2a3-43ad-a53b-300fbccf7188 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:21.941587 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab977b1d-a254-45ba-9376-8f2c9c7ef4e7 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:21.981354 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029dc5c-fa84-49b4-a823-6a255c32474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:21.981999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029dc5c-fa84-49b4-a823-6a255c32474f None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:21.982279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029dc5c-fa84-49b4-a823-6a255c32474f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:21.982553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029dc5c-fa84-49b4-a823-6a255c32474f None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:21.983229 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029dc5c-fa84-49b4-a823-6a255c32474f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:21.985587 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029dc5c-fa84-49b4-a823-6a255c32474f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:22.024564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 19/37] 10.222.0.62 () {60 vars in 1141 bytes} [Thu Sep 1 16:22:21 2022] POST /identity/v3/projects => generated 301 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:22:22.028949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029dc5c-fa84-49b4-a823-6a255c32474f None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:22.106965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddf618-d2a3-43ad-a53b-300fbccf7188 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:22.157942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03209c-9638-4184-8152-3d9c0a6034b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:22.158693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03209c-9638-4184-8152-3d9c0a6034b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:22.159018 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03209c-9638-4184-8152-3d9c0a6034b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:22.159300 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03209c-9638-4184-8152-3d9c0a6034b4 None admin] PATH_INFO: `/v3/domains/default/users/82b69c02f3fd4ddc81686f03143f0651/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:22.162318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03209c-9638-4184-8152-3d9c0a6034b4 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=82b69c02f3fd4ddc81686f03143f0651, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:22.163849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03209c-9638-4184-8152-3d9c0a6034b4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:22.198335 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029dc5c-fa84-49b4-a823-6a255c32474f None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:22.201180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:22.295089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 19/38] 10.222.0.62 () {58 vars in 1361 bytes} [Thu Sep 1 16:22:21 2022] PUT /identity/v3/domains/default/users/82b69c02f3fd4ddc81686f03143f0651/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:22:22.296939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c16664-8b15-4b1b-a4f4-fd1dc44a9683 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:22.297232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c16664-8b15-4b1b-a4f4-fd1dc44a9683 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:22.297511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c16664-8b15-4b1b-a4f4-fd1dc44a9683 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:22.302394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 20/39] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:22.308830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ebd46-cb78-479a-a44d-b966b5286b9a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:22.309198 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ebd46-cb78-479a-a44d-b966b5286b9a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:22.309424 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ebd46-cb78-479a-a44d-b966b5286b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:22.331550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b50ebd46-cb78-479a-a44d-b966b5286b9a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:22.333181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b50ebd46-cb78-479a-a44d-b966b5286b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:22.349812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:22.366039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3d844-a3ba-480d-b7c6-7d2f0b781bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:22.367529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3d844-a3ba-480d-b7c6-7d2f0b781bf8 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:22.367990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3d844-a3ba-480d-b7c6-7d2f0b781bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:22.368375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3d844-a3ba-480d-b7c6-7d2f0b781bf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:22.369285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3d844-a3ba-480d-b7c6-7d2f0b781bf8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:22.370539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3d844-a3ba-480d-b7c6-7d2f0b781bf8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:22.396176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 20/40] 10.222.0.62 () {60 vars in 1131 bytes} [Thu Sep 1 16:22:22 2022] POST /identity/v3/roles => generated 225 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:22.396176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e695c5-062b-4635-9774-0d45408af77e None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:22.396176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e695c5-062b-4635-9774-0d45408af77e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:22.396176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e695c5-062b-4635-9774-0d45408af77e None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:22.396176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 21/41] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:22.400836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1ba47-404a-4f30-a691-d89e7c751840 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:22.401168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1ba47-404a-4f30-a691-d89e7c751840 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:22.401367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1ba47-404a-4f30-a691-d89e7c751840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:22.423702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-83b1ba47-404a-4f30-a691-d89e7c751840 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:22.424310 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 21/42] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:22.425432 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-83b1ba47-404a-4f30-a691-d89e7c751840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:22.511481 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 22/43] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:22:22.690063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ebd46-cb78-479a-a44d-b966b5286b9a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:22.781295 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1ba47-404a-4f30-a691-d89e7c751840 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:22.797446 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ebd46-cb78-479a-a44d-b966b5286b9a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:22.832987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a7d7d-a767-4532-ab78-e7fefb9edc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:22.833965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a7d7d-a767-4532-ab78-e7fefb9edc01 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:22.834429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a7d7d-a767-4532-ab78-e7fefb9edc01 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:22.834901 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a7d7d-a767-4532-ab78-e7fefb9edc01 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:22.835759 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a7d7d-a767-4532-ab78-e7fefb9edc01 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:22.837766 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a7d7d-a767-4532-ab78-e7fefb9edc01 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:22.857407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 22/44] 10.222.0.62 () {60 vars in 1131 bytes} [Thu Sep 1 16:22:22 2022] POST /identity/v3/roles => generated 223 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:22.893575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:22.901455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1ba47-404a-4f30-a691-d89e7c751840 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:22.941268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd012c28-2a1a-48cb-a908-3203a139eaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:22.941838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd012c28-2a1a-48cb-a908-3203a139eaec None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:22.942117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd012c28-2a1a-48cb-a908-3203a139eaec None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:22.942389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd012c28-2a1a-48cb-a908-3203a139eaec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:22.943553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd012c28-2a1a-48cb-a908-3203a139eaec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:22.945057 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd012c28-2a1a-48cb-a908-3203a139eaec None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:22.947142 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 23/45] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:22.952816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a7d7d-a767-4532-ab78-e7fefb9edc01 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:22.965566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:23.066113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a7d7d-a767-4532-ab78-e7fefb9edc01 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:23.102118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fb7f8-1291-44e0-8539-339be8250ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:23.102736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fb7f8-1291-44e0-8539-339be8250ed3 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:23.103012 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fb7f8-1291-44e0-8539-339be8250ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:23.103278 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fb7f8-1291-44e0-8539-339be8250ed3 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:23.103960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fb7f8-1291-44e0-8539-339be8250ed3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:23.105445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fb7f8-1291-44e0-8539-339be8250ed3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:23.134628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 23/46] 10.222.0.62 () {60 vars in 1141 bytes} [Thu Sep 1 16:22:22 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:24.267918 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c80f34-db04-4234-a805-82a2966ff9ac None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:24.268464 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c80f34-db04-4234-a805-82a2966ff9ac None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:24.268464 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c80f34-db04-4234-a805-82a2966ff9ac None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:24.269844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 24/47] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:24.275331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f00361-5e87-484e-8d93-5f6bcf6f4081 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:24.275717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f00361-5e87-484e-8d93-5f6bcf6f4081 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:24.275930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f00361-5e87-484e-8d93-5f6bcf6f4081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:24.295055 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-51f00361-5e87-484e-8d93-5f6bcf6f4081 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:24.296278 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-51f00361-5e87-484e-8d93-5f6bcf6f4081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:24.355012 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:24.368523 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 24/48] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:24.633783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c80f34-db04-4234-a805-82a2966ff9ac None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:24.741227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c80f34-db04-4234-a805-82a2966ff9ac None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:24.777131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00774fa4-3150-4e0b-b36f-74e18d7830e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:24.777774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00774fa4-3150-4e0b-b36f-74e18d7830e0 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:24.778050 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00774fa4-3150-4e0b-b36f-74e18d7830e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:24.778320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00774fa4-3150-4e0b-b36f-74e18d7830e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:24.788990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00774fa4-3150-4e0b-b36f-74e18d7830e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-00774fa4-3150-4e0b-b36f-74e18d7830e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:24.794449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:24.795275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:24.796027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:24.796027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:24.796027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:24.796027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:24.796027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:24.796027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:24.796027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:24.796027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:24.796027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:24.796027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:24.796027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:24.796027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:24.796027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 25/49] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:24.801740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f00361-5e87-484e-8d93-5f6bcf6f4081 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:24.903213 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f00361-5e87-484e-8d93-5f6bcf6f4081 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:24.939536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe1461-5aed-4cf9-aca4-9d15d8d550ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:24.940195 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe1461-5aed-4cf9-aca4-9d15d8d550ca None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:24.940474 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe1461-5aed-4cf9-aca4-9d15d8d550ca None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:24.940749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe1461-5aed-4cf9-aca4-9d15d8d550ca None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:24.941416 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe1461-5aed-4cf9-aca4-9d15d8d550ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:24.941723 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe1461-5aed-4cf9-aca4-9d15d8d550ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:24.943121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe1461-5aed-4cf9-aca4-9d15d8d550ca None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:24.952302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 25/50] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:24.959056 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00774fa4-3150-4e0b-b36f-74e18d7830e0 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:24.985995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:25.030879 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326aaa4-75ed-4b44-a1cf-0141fe057a13 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:25.031167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326aaa4-75ed-4b44-a1cf-0141fe057a13 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:25.031474 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326aaa4-75ed-4b44-a1cf-0141fe057a13 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:25.032834 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 26/51] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:25.037819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24184fb6-6ade-42bb-8f67-75f50e8c3055 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:25.038105 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24184fb6-6ade-42bb-8f67-75f50e8c3055 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:25.038380 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24184fb6-6ade-42bb-8f67-75f50e8c3055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:25.065639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00774fa4-3150-4e0b-b36f-74e18d7830e0 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:25.106249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ed17e-f483-4591-99b5-594268a115e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:25.106906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ed17e-f483-4591-99b5-594268a115e3 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:25.107180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ed17e-f483-4591-99b5-594268a115e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:25.107450 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ed17e-f483-4591-99b5-594268a115e3 None admin] PATH_INFO: `/v3/users/82b69c02f3fd4ddc81686f03143f0651` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:25.108669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ed17e-f483-4591-99b5-594268a115e3 None admin] RBAC: Authorizing `identity:get_user(user_id=82b69c02f3fd4ddc81686f03143f0651)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:25.110161 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ed17e-f483-4591-99b5-594268a115e3 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:25.111802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 26/52] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:24 2022] GET /identity/v3/users/82b69c02f3fd4ddc81686f03143f0651 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:25.118299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ed17e-f483-4591-99b5-594268a115e3 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:25.166632 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-24184fb6-6ade-42bb-8f67-75f50e8c3055 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:25.167745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-24184fb6-6ade-42bb-8f67-75f50e8c3055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:25.215235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:25.225449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ed17e-f483-4591-99b5-594268a115e3 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:25.226112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:25.247793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 27/53] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:25.266555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132dd1a-9928-4d87-90a9-3ec4651a32cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:25.267220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132dd1a-9928-4d87-90a9-3ec4651a32cf None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:25.267497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132dd1a-9928-4d87-90a9-3ec4651a32cf None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:25.267764 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132dd1a-9928-4d87-90a9-3ec4651a32cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:25.269036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132dd1a-9928-4d87-90a9-3ec4651a32cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:25.270535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132dd1a-9928-4d87-90a9-3ec4651a32cf None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:25.272246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 27/54] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:25.279029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24184fb6-6ade-42bb-8f67-75f50e8c3055 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:25.378924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24184fb6-6ade-42bb-8f67-75f50e8c3055 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:25.411592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d5e11-88d2-4d57-8d2b-039d203ebc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:25.412248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d5e11-88d2-4d57-8d2b-039d203ebc9d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:25.412522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d5e11-88d2-4d57-8d2b-039d203ebc9d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:25.412788 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d5e11-88d2-4d57-8d2b-039d203ebc9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:25.413507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d5e11-88d2-4d57-8d2b-039d203ebc9d None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.domain.id=default, user.id=82b69c02f3fd4ddc81686f03143f0651) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:25.413814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d5e11-88d2-4d57-8d2b-039d203ebc9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:25.415508 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d5e11-88d2-4d57-8d2b-039d203ebc9d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:25.423431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 28/55] 10.222.0.62 () {56 vars in 1322 bytes} [Thu Sep 1 16:22:25 2022] GET /identity/v3/role_assignments?user.id=82b69c02f3fd4ddc81686f03143f0651&scope.domain.id=default&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:25.519054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132dd1a-9928-4d87-90a9-3ec4651a32cf None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:25.627814 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132dd1a-9928-4d87-90a9-3ec4651a32cf None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:25.662884 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb6d13-24a7-4441-bc4d-ae55b16d8276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:25.663519 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb6d13-24a7-4441-bc4d-ae55b16d8276 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:25.663797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb6d13-24a7-4441-bc4d-ae55b16d8276 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:25.664090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb6d13-24a7-4441-bc4d-ae55b16d8276 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:25.665440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb6d13-24a7-4441-bc4d-ae55b16d8276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:25.666906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb6d13-24a7-4441-bc4d-ae55b16d8276 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:25.668508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 28/56] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:25.674554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d5e11-88d2-4d57-8d2b-039d203ebc9d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:25.727106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:25.783580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d5e11-88d2-4d57-8d2b-039d203ebc9d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:25.821870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128a0b6-32e0-4b2f-9243-4bc5c92bf544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:25.822754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128a0b6-32e0-4b2f-9243-4bc5c92bf544 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:25.823157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128a0b6-32e0-4b2f-9243-4bc5c92bf544 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:25.823516 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128a0b6-32e0-4b2f-9243-4bc5c92bf544 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:25.824578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128a0b6-32e0-4b2f-9243-4bc5c92bf544 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:25.826833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128a0b6-32e0-4b2f-9243-4bc5c92bf544 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:25.860511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 29/57] 10.222.0.62 () {60 vars in 1140 bytes} [Thu Sep 1 16:22:25 2022] POST /identity/v3/projects => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:27.799077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba102465-245c-47b8-aa1b-b3cb5cdf20f5 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:27.799810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba102465-245c-47b8-aa1b-b3cb5cdf20f5 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:27.799955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba102465-245c-47b8-aa1b-b3cb5cdf20f5 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:27.802095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 29/58] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:27.810199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e048b-9c03-40c8-9ff2-14b92a958582 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:27.810651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e048b-9c03-40c8-9ff2-14b92a958582 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:27.810736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e048b-9c03-40c8-9ff2-14b92a958582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:27.831412 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4f7e048b-9c03-40c8-9ff2-14b92a958582 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:27.832705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4f7e048b-9c03-40c8-9ff2-14b92a958582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:27.904525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 30/59] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:22:28.180204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba102465-245c-47b8-aa1b-b3cb5cdf20f5 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:28.223153 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:28.309067 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba102465-245c-47b8-aa1b-b3cb5cdf20f5 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:28.345585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b91c94-639b-4814-8c74-95987eb01e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:28.346211 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b91c94-639b-4814-8c74-95987eb01e8f None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:28.346486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b91c94-639b-4814-8c74-95987eb01e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:28.346737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b91c94-639b-4814-8c74-95987eb01e8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:28.348000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b91c94-639b-4814-8c74-95987eb01e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:28.349451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b91c94-639b-4814-8c74-95987eb01e8f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:28.351018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 30/60] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:28.356926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e048b-9c03-40c8-9ff2-14b92a958582 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:28.385079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:28.461498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e048b-9c03-40c8-9ff2-14b92a958582 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:28.498230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29675c10-c0c2-42b0-99fa-54a516e10afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:28.498851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29675c10-c0c2-42b0-99fa-54a516e10afb None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:28.499127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29675c10-c0c2-42b0-99fa-54a516e10afb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:28.499395 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29675c10-c0c2-42b0-99fa-54a516e10afb None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:28.500071 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29675c10-c0c2-42b0-99fa-54a516e10afb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:28.501521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29675c10-c0c2-42b0-99fa-54a516e10afb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:28.536791 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 31/61] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:22:28 2022] POST /identity/v3/users => generated 275 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:31.443398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1ae27-db68-4e13-8264-2f8014609880 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:31.444264 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1ae27-db68-4e13-8264-2f8014609880 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:31.444669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1ae27-db68-4e13-8264-2f8014609880 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:31.446308 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 31/62] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:31.452024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e36f3-ba57-4562-8850-6020be0b6178 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:31.452754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e36f3-ba57-4562-8850-6020be0b6178 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:31.453163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e36f3-ba57-4562-8850-6020be0b6178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:31.476517 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9d7e36f3-ba57-4562-8850-6020be0b6178 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:31.478070 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9d7e36f3-ba57-4562-8850-6020be0b6178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:31.573241 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 32/63] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:31.815665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b17b8-fbe3-4d9a-8490-04f2338c640e None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:31.816346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b17b8-fbe3-4d9a-8490-04f2338c640e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:31.816806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b17b8-fbe3-4d9a-8490-04f2338c640e None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:31.818359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 32/64] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:31.824311 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3a5b6-c393-490b-9580-d899d3ca8c2a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:31.824831 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3a5b6-c393-490b-9580-d899d3ca8c2a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:31.825292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3a5b6-c393-490b-9580-d899d3ca8c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:31.834873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b17b8-fbe3-4d9a-8490-04f2338c640e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:31.865081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-42c3a5b6-c393-490b-9580-d899d3ca8c2a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:31.869831 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-42c3a5b6-c393-490b-9580-d899d3ca8c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:31.888491 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:31.988541 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 33/65] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:31.990817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8292575-017b-4f0b-9b9b-b1a068f3bfec None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:31.991279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8292575-017b-4f0b-9b9b-b1a068f3bfec None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:31.991407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:31.991871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8292575-017b-4f0b-9b9b-b1a068f3bfec None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:31.994242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 34/66] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:31.999759 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b5214-835f-41f3-9739-0eec3f91de09 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:32.000233 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b5214-835f-41f3-9739-0eec3f91de09 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:32.000645 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b5214-835f-41f3-9739-0eec3f91de09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:32.003767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b17b8-fbe3-4d9a-8490-04f2338c640e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:32.019988 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4c6b5214-835f-41f3-9739-0eec3f91de09 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:32.021398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4c6b5214-835f-41f3-9739-0eec3f91de09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:32.046313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746c7a8-6ef2-451a-bae8-7834a902adab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:32.047361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746c7a8-6ef2-451a-bae8-7834a902adab None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:32.047836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746c7a8-6ef2-451a-bae8-7834a902adab None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:32.048346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746c7a8-6ef2-451a-bae8-7834a902adab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:32.061082 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746c7a8-6ef2-451a-bae8-7834a902adab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-e746c7a8-6ef2-451a-bae8-7834a902adab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:32.066233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:32.067300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:32.068173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:32.068173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:32.068173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:32.068173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:32.068173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:32.068173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:32.068173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:32.068173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:32.068173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:32.068173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:32.068173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:32.068173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:32.070582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 33/67] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:32.072145 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2305f24-2dd3-48a2-8af3-893e0c76ac3f None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:32.072674 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2305f24-2dd3-48a2-8af3-893e0c76ac3f None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:32.073137 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2305f24-2dd3-48a2-8af3-893e0c76ac3f None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:32.075575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 34/68] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:32.076586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2305f24-2dd3-48a2-8af3-893e0c76ac3f None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:32.107025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 35/69] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:32.109310 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd1f57-00d6-4e10-a7d8-319d89f5aa76 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:32.109594 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd1f57-00d6-4e10-a7d8-319d89f5aa76 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:32.109864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd1f57-00d6-4e10-a7d8-319d89f5aa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:32.145185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-67bd1f57-00d6-4e10-a7d8-319d89f5aa76 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:32.147799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-67bd1f57-00d6-4e10-a7d8-319d89f5aa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:32.241125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2305f24-2dd3-48a2-8af3-893e0c76ac3f None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:32.286143 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 36/70] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:32.303630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f695fa-ea83-469d-a105-0b306370c97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:32.305618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd1f57-00d6-4e10-a7d8-319d89f5aa76 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:32.306347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f695fa-ea83-469d-a105-0b306370c97f None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:32.306685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f695fa-ea83-469d-a105-0b306370c97f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:32.306955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f695fa-ea83-469d-a105-0b306370c97f None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:32.307680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f695fa-ea83-469d-a105-0b306370c97f None admin] RBAC: Adding query filter params (name=member) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:32.308014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f695fa-ea83-469d-a105-0b306370c97f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:32.309782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f695fa-ea83-469d-a105-0b306370c97f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:32.311557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:32.323696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 35/71] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:32.334838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f695fa-ea83-469d-a105-0b306370c97f None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:32.367120 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:32.470958 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd1f57-00d6-4e10-a7d8-319d89f5aa76 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:32.480637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f695fa-ea83-469d-a105-0b306370c97f None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:32.506053 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2618e908-6ec6-4992-af83-48aabafe4b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:32.506842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618e908-6ec6-4992-af83-48aabafe4b55 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:32.507126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618e908-6ec6-4992-af83-48aabafe4b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:32.507482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618e908-6ec6-4992-af83-48aabafe4b55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:32.517648 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907caf73-7354-4793-90f3-5b1a044355e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:32.518284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907caf73-7354-4793-90f3-5b1a044355e8 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:32.518559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907caf73-7354-4793-90f3-5b1a044355e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:32.518827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907caf73-7354-4793-90f3-5b1a044355e8 None admin] PATH_INFO: `/v3/users/ab56ac1625ce49dfba98e1bb018da72d` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:32.520214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2618e908-6ec6-4992-af83-48aabafe4b55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2618e908-6ec6-4992-af83-48aabafe4b55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:32.526808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:32.527796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:32.528798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:32.528798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:32.528798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:32.528798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:32.528798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:32.528798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:32.528798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:32.528798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:32.528798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:32.528798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:32.528798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:32.528798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:32.528798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 37/72] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:32.529738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05ef06-920a-4e9d-bf1e-1510b6ed9313 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:32.529738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05ef06-920a-4e9d-bf1e-1510b6ed9313 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:32.529738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05ef06-920a-4e9d-bf1e-1510b6ed9313 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:32.530855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 38/73] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:32.531540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05ef06-920a-4e9d-bf1e-1510b6ed9313 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:32.539867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907caf73-7354-4793-90f3-5b1a044355e8 None admin] RBAC: Authorizing `identity:get_user(user_id=ab56ac1625ce49dfba98e1bb018da72d)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:32.541405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907caf73-7354-4793-90f3-5b1a044355e8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:32.548085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 36/74] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:32 2022] GET /identity/v3/users/ab56ac1625ce49dfba98e1bb018da72d => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:32.548085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907caf73-7354-4793-90f3-5b1a044355e8 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:32.601660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:32.625474 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:32.640993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05ef06-920a-4e9d-bf1e-1510b6ed9313 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:32.653408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907caf73-7354-4793-90f3-5b1a044355e8 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:32.674712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c8770-5c84-4fba-8a9a-5dcbba1f30bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:32.675357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c8770-5c84-4fba-8a9a-5dcbba1f30bc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:32.675636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c8770-5c84-4fba-8a9a-5dcbba1f30bc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:32.675940 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c8770-5c84-4fba-8a9a-5dcbba1f30bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:32.686345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c8770-5c84-4fba-8a9a-5dcbba1f30bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:32.687458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedaeb7-5bec-4cfd-963f-9274cbb4d9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:32.688102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedaeb7-5bec-4cfd-963f-9274cbb4d9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:32.688384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedaeb7-5bec-4cfd-963f-9274cbb4d9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:32.688654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedaeb7-5bec-4cfd-963f-9274cbb4d9d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:32.689315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedaeb7-5bec-4cfd-963f-9274cbb4d9d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:32.689621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedaeb7-5bec-4cfd-963f-9274cbb4d9d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:32.691020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedaeb7-5bec-4cfd-963f-9274cbb4d9d1 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2e3c8770-5c84-4fba-8a9a-5dcbba1f30bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:32.691254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:32.692222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:32.692965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:32.692965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:32.692965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:32.692965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:32.692965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:32.692965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:32.692965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:32.692965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:32.692965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:32.692965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:32.692965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:22:32.692965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:32.692965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 39/75] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:22:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:32.693960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1c756-15b7-4b88-8c07-a39f0a31aac4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:32.694226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1c756-15b7-4b88-8c07-a39f0a31aac4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:32.694492 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1c756-15b7-4b88-8c07-a39f0a31aac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:32.708319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 37/76] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:32.708319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedaeb7-5bec-4cfd-963f-9274cbb4d9d1 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:32.714469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d2f1c756-15b7-4b88-8c07-a39f0a31aac4 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:32.715699 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d2f1c756-15b7-4b88-8c07-a39f0a31aac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:32.783578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 40/77] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:32.785096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1c756-15b7-4b88-8c07-a39f0a31aac4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:32.810009 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedaeb7-5bec-4cfd-963f-9274cbb4d9d1 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:32.845458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc7601-757a-4b9f-ac54-ac2bbca897c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:32.846072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc7601-757a-4b9f-ac54-ac2bbca897c0 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:32.846335 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc7601-757a-4b9f-ac54-ac2bbca897c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:32.846602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc7601-757a-4b9f-ac54-ac2bbca897c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:32.847532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:32.850864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:32.893551 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1c756-15b7-4b88-8c07-a39f0a31aac4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:32.919045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc7601-757a-4b9f-ac54-ac2bbca897c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:32.920591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc7601-757a-4b9f-ac54-ac2bbca897c0 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:32.929372 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a7fa4-02e7-44ec-b7f7-453ff3246111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:32.930363 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a7fa4-02e7-44ec-b7f7-453ff3246111 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:32.930817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a7fa4-02e7-44ec-b7f7-453ff3246111 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:32.931283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a7fa4-02e7-44ec-b7f7-453ff3246111 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:32.942467 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a7fa4-02e7-44ec-b7f7-453ff3246111 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8f2a7fa4-02e7-44ec-b7f7-453ff3246111 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:32.948298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:32.949139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:32.949917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:32.949917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:32.949917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:32.949917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:32.949917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:32.949917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:32.949917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:32.949917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:32.949917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:32.949917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:32.949917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:32.949917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:32.951860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 41/78] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:32.956990 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a7fa4-02e7-44ec-b7f7-453ff3246111 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:33.022338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 38/79] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:33.023194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc7601-757a-4b9f-ac54-ac2bbca897c0 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:33.060656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:33.062878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a7fa4-02e7-44ec-b7f7-453ff3246111 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:33.077654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:33.098318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25bbd4-fee5-47f0-9d1c-e417beba003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:33.098932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25bbd4-fee5-47f0-9d1c-e417beba003a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:33.099210 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25bbd4-fee5-47f0-9d1c-e417beba003a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:33.099477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25bbd4-fee5-47f0-9d1c-e417beba003a None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:33.100153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc25bbd4-fee5-47f0-9d1c-e417beba003a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:33.100471 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc25bbd4-fee5-47f0-9d1c-e417beba003a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:33.101872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc25bbd4-fee5-47f0-9d1c-e417beba003a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:33.109950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 42/80] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:33.110833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25bbd4-fee5-47f0-9d1c-e417beba003a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:33.126324 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc7601-757a-4b9f-ac54-ac2bbca897c0 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:33.160214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86d432-fb9f-44a9-a8f4-219b9b2f7a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:33.160868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86d432-fb9f-44a9-a8f4-219b9b2f7a3a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:33.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86d432-fb9f-44a9-a8f4-219b9b2f7a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:33.161433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86d432-fb9f-44a9-a8f4-219b9b2f7a3a None admin] PATH_INFO: `/v3/users/82b69c02f3fd4ddc81686f03143f0651` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:33.162577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86d432-fb9f-44a9-a8f4-219b9b2f7a3a None admin] RBAC: Authorizing `identity:get_user(user_id=82b69c02f3fd4ddc81686f03143f0651)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:33.164103 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86d432-fb9f-44a9-a8f4-219b9b2f7a3a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:33.165773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 39/81] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:33 2022] GET /identity/v3/users/82b69c02f3fd4ddc81686f03143f0651 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:33.168775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86d432-fb9f-44a9-a8f4-219b9b2f7a3a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:33.211905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25bbd4-fee5-47f0-9d1c-e417beba003a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:33.245295 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea40a1f-3d31-4365-b82a-c4946ebf1b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:33.245914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea40a1f-3d31-4365-b82a-c4946ebf1b56 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:33.246190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea40a1f-3d31-4365-b82a-c4946ebf1b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:33.246472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea40a1f-3d31-4365-b82a-c4946ebf1b56 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:33.247147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea40a1f-3d31-4365-b82a-c4946ebf1b56 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:33.247444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea40a1f-3d31-4365-b82a-c4946ebf1b56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:33.248851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea40a1f-3d31-4365-b82a-c4946ebf1b56 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:33.256771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 43/82] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:33.257559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea40a1f-3d31-4365-b82a-c4946ebf1b56 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:33.269001 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86d432-fb9f-44a9-a8f4-219b9b2f7a3a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:33.271264 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:33.296727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:33.304760 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815915cb-ccbf-4875-9d69-1dd1986aacb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:33.305502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815915cb-ccbf-4875-9d69-1dd1986aacb6 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:33.305783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815915cb-ccbf-4875-9d69-1dd1986aacb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:33.306178 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815915cb-ccbf-4875-9d69-1dd1986aacb6 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:33.307305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815915cb-ccbf-4875-9d69-1dd1986aacb6 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:33.308795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815915cb-ccbf-4875-9d69-1dd1986aacb6 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:33.310470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 40/83] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:33 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:33.311333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815915cb-ccbf-4875-9d69-1dd1986aacb6 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:33.358148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea40a1f-3d31-4365-b82a-c4946ebf1b56 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:33.391883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdeb0e9-bf71-4bf7-a8bd-2af1a588b36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:33.392550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdeb0e9-bf71-4bf7-a8bd-2af1a588b36c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:33.392829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdeb0e9-bf71-4bf7-a8bd-2af1a588b36c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:33.393096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdeb0e9-bf71-4bf7-a8bd-2af1a588b36c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:33.394194 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdeb0e9-bf71-4bf7-a8bd-2af1a588b36c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:33.395639 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdeb0e9-bf71-4bf7-a8bd-2af1a588b36c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:33.397231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 44/84] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:33.397996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdeb0e9-bf71-4bf7-a8bd-2af1a588b36c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:33.418903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815915cb-ccbf-4875-9d69-1dd1986aacb6 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:33.454988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b79c42-0239-45f6-8459-08febbfff901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:33.455641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b79c42-0239-45f6-8459-08febbfff901 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:33.455949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b79c42-0239-45f6-8459-08febbfff901 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:33.456226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b79c42-0239-45f6-8459-08febbfff901 None admin] PATH_INFO: `/v3/users/ab56ac1625ce49dfba98e1bb018da72d` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:33.457395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b79c42-0239-45f6-8459-08febbfff901 None admin] RBAC: Authorizing `identity:get_user(user_id=ab56ac1625ce49dfba98e1bb018da72d)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:33.458881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b79c42-0239-45f6-8459-08febbfff901 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:33.460546 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 41/85] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:33 2022] GET /identity/v3/users/ab56ac1625ce49dfba98e1bb018da72d => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:33.461431 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b79c42-0239-45f6-8459-08febbfff901 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:33.502823 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:33.508574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdeb0e9-bf71-4bf7-a8bd-2af1a588b36c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:33.516598 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:33.547916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17633e5c-aa9b-4fa6-afbd-9b9118e68d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:33.548555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17633e5c-aa9b-4fa6-afbd-9b9118e68d60 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:33.548835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17633e5c-aa9b-4fa6-afbd-9b9118e68d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:33.549112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17633e5c-aa9b-4fa6-afbd-9b9118e68d60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:33.565526 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b79c42-0239-45f6-8459-08febbfff901 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:33.602670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb7c3c-dc75-432d-85fe-bd8089905106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:33.603289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb7c3c-dc75-432d-85fe-bd8089905106 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:33.603577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb7c3c-dc75-432d-85fe-bd8089905106 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:33.603832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb7c3c-dc75-432d-85fe-bd8089905106 None admin] PATH_INFO: `/v3/users/ab56ac1625ce49dfba98e1bb018da72d` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:33.604974 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb7c3c-dc75-432d-85fe-bd8089905106 None admin] RBAC: Authorizing `identity:get_user(user_id=ab56ac1625ce49dfba98e1bb018da72d)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:33.606430 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb7c3c-dc75-432d-85fe-bd8089905106 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:33.608041 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 42/86] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:33 2022] GET /identity/v3/users/ab56ac1625ce49dfba98e1bb018da72d => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:33.608800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb7c3c-dc75-432d-85fe-bd8089905106 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:33.625013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17633e5c-aa9b-4fa6-afbd-9b9118e68d60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:33.626522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17633e5c-aa9b-4fa6-afbd-9b9118e68d60 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:33.712575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb7c3c-dc75-432d-85fe-bd8089905106 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:33.725992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:33.731798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 45/87] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:33.731798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17633e5c-aa9b-4fa6-afbd-9b9118e68d60 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:33.736088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:33.753910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eccb78d-22b6-409e-b768-60185de9fb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:33.754526 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eccb78d-22b6-409e-b768-60185de9fb19 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:33.754808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eccb78d-22b6-409e-b768-60185de9fb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:33.755082 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eccb78d-22b6-409e-b768-60185de9fb19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:33.756301 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eccb78d-22b6-409e-b768-60185de9fb19 None admin] RBAC: Adding query filter params (role.id=3100f6054f7d4d7b984fe19a4d8ba291, scope.project.id=ce0bf520fc554803bd196782f428d532, user.id=ab56ac1625ce49dfba98e1bb018da72d) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:33.756616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eccb78d-22b6-409e-b768-60185de9fb19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:33.758106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eccb78d-22b6-409e-b768-60185de9fb19 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:33.764117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 43/88] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:33 2022] GET /identity/v3/role_assignments?user.id=ab56ac1625ce49dfba98e1bb018da72d&scope.project.id=ce0bf520fc554803bd196782f428d532&role.id=3100f6054f7d4d7b984fe19a4d8ba291 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:33.764930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eccb78d-22b6-409e-b768-60185de9fb19 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:33.836405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17633e5c-aa9b-4fa6-afbd-9b9118e68d60 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:33.874045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2691d-8800-4888-b248-035b953f1a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:33.874678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2691d-8800-4888-b248-035b953f1a8a None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:33.874782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eccb78d-22b6-409e-b768-60185de9fb19 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:33.874958 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2691d-8800-4888-b248-035b953f1a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:33.875227 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2691d-8800-4888-b248-035b953f1a8a None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:33.875903 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2691d-8800-4888-b248-035b953f1a8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:33.878513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2691d-8800-4888-b248-035b953f1a8a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:33.915918 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 46/89] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:22:33 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:33.915918 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2691d-8800-4888-b248-035b953f1a8a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:33.929942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294c12b-6a05-4105-ad12-2cc909095ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:33.930701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294c12b-6a05-4105-ad12-2cc909095ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:33.930942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294c12b-6a05-4105-ad12-2cc909095ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:33.931388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294c12b-6a05-4105-ad12-2cc909095ab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:33.996637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:33.999761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:34.040291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294c12b-6a05-4105-ad12-2cc909095ab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:34.042761 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294c12b-6a05-4105-ad12-2cc909095ab9 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:34.056833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2691d-8800-4888-b248-035b953f1a8a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:34.101807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f04ce1-8318-44ad-8984-6afe5ab832ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:34.102540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f04ce1-8318-44ad-8984-6afe5ab832ae None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:34.102926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f04ce1-8318-44ad-8984-6afe5ab832ae None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:34.103190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f04ce1-8318-44ad-8984-6afe5ab832ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:34.172806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 44/90] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:34.173492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294c12b-6a05-4105-ad12-2cc909095ab9 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:34.199576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f04ce1-8318-44ad-8984-6afe5ab832ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:34.201591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f04ce1-8318-44ad-8984-6afe5ab832ae None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:34.259787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:34.285203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:34.300132 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294c12b-6a05-4105-ad12-2cc909095ab9 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:34.341661 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5429509-e234-4d5a-b444-243d4299cf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:34.342362 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5429509-e234-4d5a-b444-243d4299cf95 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:34.342641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5429509-e234-4d5a-b444-243d4299cf95 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:34.342908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5429509-e234-4d5a-b444-243d4299cf95 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:34.344147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5429509-e234-4d5a-b444-243d4299cf95 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:34.345153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 47/91] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:34.345716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5429509-e234-4d5a-b444-243d4299cf95 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:34.346583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f04ce1-8318-44ad-8984-6afe5ab832ae None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:34.347616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 45/92] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:34 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:34.353768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5429509-e234-4d5a-b444-243d4299cf95 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:34.467734 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5429509-e234-4d5a-b444-243d4299cf95 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:34.504408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37decb-cc8c-44d8-9ce3-29e4efc6e75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:34.505063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37decb-cc8c-44d8-9ce3-29e4efc6e75d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:34.505340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37decb-cc8c-44d8-9ce3-29e4efc6e75d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:34.505612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37decb-cc8c-44d8-9ce3-29e4efc6e75d None admin] PATH_INFO: `/v3/projects/161024fa80304870ad5a26ea4c23b96b` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:34.506839 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37decb-cc8c-44d8-9ce3-29e4efc6e75d None admin] RBAC: Authorizing `identity:get_project(project_id=161024fa80304870ad5a26ea4c23b96b)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:34.508534 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37decb-cc8c-44d8-9ce3-29e4efc6e75d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:34.510392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 46/93] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:34 2022] GET /identity/v3/projects/161024fa80304870ad5a26ea4c23b96b => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:34.511078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37decb-cc8c-44d8-9ce3-29e4efc6e75d None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:34.522274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:34.561992 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f04ce1-8318-44ad-8984-6afe5ab832ae None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:34.617449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37decb-cc8c-44d8-9ce3-29e4efc6e75d None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:34.651804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a261e8c-8ba5-441d-8c84-1cd05acbf0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:34.652454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a261e8c-8ba5-441d-8c84-1cd05acbf0ea None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:34.652729 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a261e8c-8ba5-441d-8c84-1cd05acbf0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:34.652997 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a261e8c-8ba5-441d-8c84-1cd05acbf0ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:34.653722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a261e8c-8ba5-441d-8c84-1cd05acbf0ea None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=ce0bf520fc554803bd196782f428d532, user.id=82b69c02f3fd4ddc81686f03143f0651) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:34.654023 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a261e8c-8ba5-441d-8c84-1cd05acbf0ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:34.655483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a261e8c-8ba5-441d-8c84-1cd05acbf0ea None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:34.660624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68950aa-a07d-4317-a386-079317f80286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:34.661127 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 47/94] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:34 2022] GET /identity/v3/role_assignments?user.id=82b69c02f3fd4ddc81686f03143f0651&scope.project.id=ce0bf520fc554803bd196782f428d532&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:34.661626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68950aa-a07d-4317-a386-079317f80286 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:34.661909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68950aa-a07d-4317-a386-079317f80286 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:34.662006 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a261e8c-8ba5-441d-8c84-1cd05acbf0ea None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:34.662187 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68950aa-a07d-4317-a386-079317f80286 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:34.663934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68950aa-a07d-4317-a386-079317f80286 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:34.665621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68950aa-a07d-4317-a386-079317f80286 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:34.667576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 48/95] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:34 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:34.676900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68950aa-a07d-4317-a386-079317f80286 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:34.742810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:34.808448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:34.879060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68950aa-a07d-4317-a386-079317f80286 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:34.902013 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a261e8c-8ba5-441d-8c84-1cd05acbf0ea None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:34.947116 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d8292-c2b0-4e27-954a-0f886838fbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:34.948033 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d8292-c2b0-4e27-954a-0f886838fbad None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:34.948608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d8292-c2b0-4e27-954a-0f886838fbad None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:34.950215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d8292-c2b0-4e27-954a-0f886838fbad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:34.951101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d8292-c2b0-4e27-954a-0f886838fbad None admin] RBAC: Adding query filter params (role.id=be3ffa4db9954414868f116f97f3a929, scope.project.id=ce0bf520fc554803bd196782f428d532, user.id=ab56ac1625ce49dfba98e1bb018da72d) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:34.952249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d8292-c2b0-4e27-954a-0f886838fbad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:34.953859 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d8292-c2b0-4e27-954a-0f886838fbad None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:34.964561 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 49/96] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:34 2022] GET /identity/v3/role_assignments?user.id=ab56ac1625ce49dfba98e1bb018da72d&scope.project.id=ce0bf520fc554803bd196782f428d532&role.id=be3ffa4db9954414868f116f97f3a929 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:34.978596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6424fe7-5af9-48af-b604-2e22199afeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:34.979522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6424fe7-5af9-48af-b604-2e22199afeb9 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:34.979966 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6424fe7-5af9-48af-b604-2e22199afeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:34.980374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6424fe7-5af9-48af-b604-2e22199afeb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:34.981264 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6424fe7-5af9-48af-b604-2e22199afeb9 None admin] RBAC: Adding query filter params (role.id=3100f6054f7d4d7b984fe19a4d8ba291, scope.project.id=161024fa80304870ad5a26ea4c23b96b, user.id=ab56ac1625ce49dfba98e1bb018da72d) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:34.981706 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6424fe7-5af9-48af-b604-2e22199afeb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:34.983385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6424fe7-5af9-48af-b604-2e22199afeb9 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:34.993048 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 48/97] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:34 2022] GET /identity/v3/role_assignments?user.id=ab56ac1625ce49dfba98e1bb018da72d&scope.project.id=161024fa80304870ad5a26ea4c23b96b&role.id=3100f6054f7d4d7b984fe19a4d8ba291 => generated 259 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:36.092401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cec34-aaa3-468d-8a0c-a56d79fc5c84 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:36.093428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cec34-aaa3-468d-8a0c-a56d79fc5c84 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:36.093918 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cec34-aaa3-468d-8a0c-a56d79fc5c84 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:36.095456 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 50/98] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:36.100876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e661d-3721-4479-8014-1142c19d4d4e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:36.101645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e661d-3721-4479-8014-1142c19d4d4e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:36.102307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e661d-3721-4479-8014-1142c19d4d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:36.128855 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-0d3e661d-3721-4479-8014-1142c19d4d4e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:36.130376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0d3e661d-3721-4479-8014-1142c19d4d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:36.211503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 49/99] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:36.736127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cec34-aaa3-468d-8a0c-a56d79fc5c84 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:36.814857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c40a8e-2259-4765-b596-16b252551a5c None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:36.815467 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c40a8e-2259-4765-b596-16b252551a5c None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:36.815941 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c40a8e-2259-4765-b596-16b252551a5c None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:36.817481 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 50/100] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:22:36.825812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e1f78-24f6-425e-b52f-03fb5e3005b9 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:36.826350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e1f78-24f6-425e-b52f-03fb5e3005b9 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:36.826831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e1f78-24f6-425e-b52f-03fb5e3005b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:36.850686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-189e1f78-24f6-425e-b52f-03fb5e3005b9 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:36.853430 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-189e1f78-24f6-425e-b52f-03fb5e3005b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:36.912185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:36.921862 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cec34-aaa3-468d-8a0c-a56d79fc5c84 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:36.976485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:36.989476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 51/101] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:22:36.991027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326060f-4982-4017-b5fa-b8b18f80d1fa None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:36.991977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326060f-4982-4017-b5fa-b8b18f80d1fa None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:36.992802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f498e92-22cc-46d1-b2ae-4637fd03e3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:36.993619 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f498e92-22cc-46d1-b2ae-4637fd03e3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:36.994028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f498e92-22cc-46d1-b2ae-4637fd03e3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:36.994423 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f498e92-22cc-46d1-b2ae-4637fd03e3a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:36.995302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326060f-4982-4017-b5fa-b8b18f80d1fa None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:36.996922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 52/102] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:37.003039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f43e30-22c0-47ab-97a3-4281092e4388 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:37.003504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f43e30-22c0-47ab-97a3-4281092e4388 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:37.003945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f43e30-22c0-47ab-97a3-4281092e4388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:37.016377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f498e92-22cc-46d1-b2ae-4637fd03e3a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8f498e92-22cc-46d1-b2ae-4637fd03e3a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:37.021524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:37.022537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:37.023450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:37.023450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:37.023450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:37.023450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:37.023450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:37.023450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:37.023450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:37.023450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:37.023450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:37.023450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:37.023450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:37.023450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:37.029477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 51/103] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:37.037644 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f498e92-22cc-46d1-b2ae-4637fd03e3a4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:37.041593 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-c5f43e30-22c0-47ab-97a3-4281092e4388 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:37.042929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c5f43e30-22c0-47ab-97a3-4281092e4388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:37.111008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 53/104] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:22:37.199096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f498e92-22cc-46d1-b2ae-4637fd03e3a4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:37.244062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71e7b1-0a8c-47ab-a7cc-e06367c8eca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:37.244663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71e7b1-0a8c-47ab-a7cc-e06367c8eca7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:37.244950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71e7b1-0a8c-47ab-a7cc-e06367c8eca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:37.245225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71e7b1-0a8c-47ab-a7cc-e06367c8eca7 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:37.245900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71e7b1-0a8c-47ab-a7cc-e06367c8eca7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:37.246212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71e7b1-0a8c-47ab-a7cc-e06367c8eca7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:37.247614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71e7b1-0a8c-47ab-a7cc-e06367c8eca7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:37.256551 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 52/105] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:37.265029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f43e30-22c0-47ab-97a3-4281092e4388 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:37.373037 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f43e30-22c0-47ab-97a3-4281092e4388 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:37.399459 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:37.412609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6620e9-d34a-4257-991d-fa6b3fc16587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:37.413259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6620e9-d34a-4257-991d-fa6b3fc16587 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:37.413532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6620e9-d34a-4257-991d-fa6b3fc16587 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:37.413798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6620e9-d34a-4257-991d-fa6b3fc16587 None admin] PATH_INFO: `/v3/users/ab56ac1625ce49dfba98e1bb018da72d` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:37.415080 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6620e9-d34a-4257-991d-fa6b3fc16587 None admin] RBAC: Authorizing `identity:get_user(user_id=ab56ac1625ce49dfba98e1bb018da72d)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:37.416600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6620e9-d34a-4257-991d-fa6b3fc16587 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:37.418403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 54/106] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:37 2022] GET /identity/v3/users/ab56ac1625ce49dfba98e1bb018da72d => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:37.423843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71e7b1-0a8c-47ab-a7cc-e06367c8eca7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:37.453968 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:37.503085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6620e9-d34a-4257-991d-fa6b3fc16587 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:37.580533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71e7b1-0a8c-47ab-a7cc-e06367c8eca7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:37.609133 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6620e9-d34a-4257-991d-fa6b3fc16587 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:37.643129 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704bd8e-f1c8-4f7e-823c-f3d587e0066f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:37.643907 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704bd8e-f1c8-4f7e-823c-f3d587e0066f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:37.644187 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704bd8e-f1c8-4f7e-823c-f3d587e0066f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:37.644463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704bd8e-f1c8-4f7e-823c-f3d587e0066f None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:37.645876 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704bd8e-f1c8-4f7e-823c-f3d587e0066f None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:37.646830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7810c1-4cdd-4b84-bc6c-9d2724714f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:37.647453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704bd8e-f1c8-4f7e-823c-f3d587e0066f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:37.647910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7810c1-4cdd-4b84-bc6c-9d2724714f4f None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:37.648372 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7810c1-4cdd-4b84-bc6c-9d2724714f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:37.648835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7810c1-4cdd-4b84-bc6c-9d2724714f4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:37.655929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 53/107] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:37 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:37.656865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704bd8e-f1c8-4f7e-823c-f3d587e0066f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:37.657987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7810c1-4cdd-4b84-bc6c-9d2724714f4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:37.659469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7810c1-4cdd-4b84-bc6c-9d2724714f4f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:37.661909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 55/108] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:37.662679 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7810c1-4cdd-4b84-bc6c-9d2724714f4f None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:37.782555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:37.808257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:37.825283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7810c1-4cdd-4b84-bc6c-9d2724714f4f None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:37.827610 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704bd8e-f1c8-4f7e-823c-f3d587e0066f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:37.862341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8179b1-926e-4814-90e6-7d883145865c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:37.863042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8179b1-926e-4814-90e6-7d883145865c None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:37.863329 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8179b1-926e-4814-90e6-7d883145865c None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:37.863610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8179b1-926e-4814-90e6-7d883145865c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:37.864747 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d25323-482c-4258-9fb3-bee66f3fa3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:37.865420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d25323-482c-4258-9fb3-bee66f3fa3c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:37.865701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d25323-482c-4258-9fb3-bee66f3fa3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:37.865970 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d25323-482c-4258-9fb3-bee66f3fa3c7 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532/users/ab56ac1625ce49dfba98e1bb018da72d/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:37.868060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d25323-482c-4258-9fb3-bee66f3fa3c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce0bf520fc554803bd196782f428d532, user_id=ab56ac1625ce49dfba98e1bb018da72d, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:37.869620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d25323-482c-4258-9fb3-bee66f3fa3c7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:37.875504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8179b1-926e-4814-90e6-7d883145865c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-fb8179b1-926e-4814-90e6-7d883145865c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:37.880597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:37.881635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:37.882577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:37.882577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:37.882577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:37.882577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:37.882577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:37.882577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:37.882577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:37.882577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:37.882577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:37.882577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:37.882577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:22:37.882577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:37.884834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 56/109] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:37.885739 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8179b1-926e-4814-90e6-7d883145865c None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:37.886725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 54/110] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:22:37 2022] PUT /identity/v3/projects/ce0bf520fc554803bd196782f428d532/users/ab56ac1625ce49dfba98e1bb018da72d/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:22:37.895671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9d5e8-1c29-448e-978e-fa9dfe357339 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:37.895671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9d5e8-1c29-448e-978e-fa9dfe357339 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:37.895671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9d5e8-1c29-448e-978e-fa9dfe357339 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:37.895671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 55/111] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:37.897604 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921fb0b-6089-42d4-b383-0f96df3c2339 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:37.897725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921fb0b-6089-42d4-b383-0f96df3c2339 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:37.897970 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921fb0b-6089-42d4-b383-0f96df3c2339 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:37.899773 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 56/112] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:37.900613 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921fb0b-6089-42d4-b383-0f96df3c2339 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:38.006623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8179b1-926e-4814-90e6-7d883145865c None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:38.018899 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921fb0b-6089-42d4-b383-0f96df3c2339 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:38.034703 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:38.048959 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53cc944-0387-4d06-805b-38a9b9d75296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:38.049843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cc944-0387-4d06-805b-38a9b9d75296 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:38.050266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cc944-0387-4d06-805b-38a9b9d75296 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:38.050680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cc944-0387-4d06-805b-38a9b9d75296 None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:38.051528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53cc944-0387-4d06-805b-38a9b9d75296 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:38.053057 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53cc944-0387-4d06-805b-38a9b9d75296 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:38.059361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d4fa0-f05a-4ba9-b6ec-09c7f82d02b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:38.060010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d4fa0-f05a-4ba9-b6ec-09c7f82d02b1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:38.060283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d4fa0-f05a-4ba9-b6ec-09c7f82d02b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:38.060550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d4fa0-f05a-4ba9-b6ec-09c7f82d02b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:38.061209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d4fa0-f05a-4ba9-b6ec-09c7f82d02b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:38.061503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d4fa0-f05a-4ba9-b6ec-09c7f82d02b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:38.062885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d4fa0-f05a-4ba9-b6ec-09c7f82d02b1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:38.072159 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 57/113] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:38.073391 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe043cc-ec5c-4b40-aa84-66fb214f4b0f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:38.073674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe043cc-ec5c-4b40-aa84-66fb214f4b0f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:38.073948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe043cc-ec5c-4b40-aa84-66fb214f4b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:38.082874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 57/114] 10.222.0.62 () {60 vars in 1141 bytes} [Thu Sep 1 16:22:37 2022] POST /identity/v3/projects => generated 302 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:38.084303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d581c05-a1e3-4932-bd3f-3e43cf62ec4a None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:38.084595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d581c05-a1e3-4932-bd3f-3e43cf62ec4a None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:38.084870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d581c05-a1e3-4932-bd3f-3e43cf62ec4a None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:38.086768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 58/115] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:38.089019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95154c61-ea29-4ebd-bb8d-68bfd0506642 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:38.089306 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95154c61-ea29-4ebd-bb8d-68bfd0506642 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:38.089585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95154c61-ea29-4ebd-bb8d-68bfd0506642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:38.100701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:38.104960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bbe043cc-ec5c-4b40-aa84-66fb214f4b0f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:38.106675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bbe043cc-ec5c-4b40-aa84-66fb214f4b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:38.111201 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-95154c61-ea29-4ebd-bb8d-68bfd0506642 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:38.111745 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-95154c61-ea29-4ebd-bb8d-68bfd0506642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:38.187798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 59/116] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:38.188778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95154c61-ea29-4ebd-bb8d-68bfd0506642 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:38.190712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 58/117] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:38.197192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4bc5b-a821-47f1-aeb7-93d4aa5365f4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:38.197911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4bc5b-a821-47f1-aeb7-93d4aa5365f4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:38.198512 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4bc5b-a821-47f1-aeb7-93d4aa5365f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:38.233003 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-31b4bc5b-a821-47f1-aeb7-93d4aa5365f4 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:38.235010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-31b4bc5b-a821-47f1-aeb7-93d4aa5365f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:38.362292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95154c61-ea29-4ebd-bb8d-68bfd0506642 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:38.376271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:38.379360 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 59/118] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:38.420045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb825e-5eb6-4094-b20e-1331a6b90569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:38.420764 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb825e-5eb6-4094-b20e-1331a6b90569 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:38.421050 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb825e-5eb6-4094-b20e-1331a6b90569 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:38.421323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb825e-5eb6-4094-b20e-1331a6b90569 None admin] PATH_INFO: `/v3/users/6e9ec43d2aed457589dcc8a3620749d5` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:38.443249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb825e-5eb6-4094-b20e-1331a6b90569 None admin] RBAC: Authorizing `identity:get_user(user_id=6e9ec43d2aed457589dcc8a3620749d5)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:38.447611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb825e-5eb6-4094-b20e-1331a6b90569 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:38.450046 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 60/119] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:38 2022] GET /identity/v3/users/6e9ec43d2aed457589dcc8a3620749d5 => generated 289 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:38.456444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4bc5b-a821-47f1-aeb7-93d4aa5365f4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:38.512824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb825e-5eb6-4094-b20e-1331a6b90569 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:38.552223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:38.607285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4bc5b-a821-47f1-aeb7-93d4aa5365f4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:38.669133 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb825e-5eb6-4094-b20e-1331a6b90569 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:38.684530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc1d58-313b-4fd3-aacc-03e6edcffbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:38.685173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc1d58-313b-4fd3-aacc-03e6edcffbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:38.685455 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc1d58-313b-4fd3-aacc-03e6edcffbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:38.685729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc1d58-313b-4fd3-aacc-03e6edcffbb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:38.716609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f993b41-9a8d-496a-aecc-8bfd6744e0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:38.717263 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f993b41-9a8d-496a-aecc-8bfd6744e0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:38.717538 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f993b41-9a8d-496a-aecc-8bfd6744e0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:38.717804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f993b41-9a8d-496a-aecc-8bfd6744e0d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:38.733198 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f993b41-9a8d-496a-aecc-8bfd6744e0d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-1f993b41-9a8d-496a-aecc-8bfd6744e0d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:38.744379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:38.745242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:38.746580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:38.746580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:38.746580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:38.746580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:38.746580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:38.746580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:38.746580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:38.746580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:38.746580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:38.746580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:38.746580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:38.746580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:38.746580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 61/120] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:38.746580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f993b41-9a8d-496a-aecc-8bfd6744e0d3 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:38.758382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc1d58-313b-4fd3-aacc-03e6edcffbb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:38.758382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc1d58-313b-4fd3-aacc-03e6edcffbb4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:38.789758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:38.831166 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:38.870978 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 60/121] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:38.872086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc1d58-313b-4fd3-aacc-03e6edcffbb4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:38.874280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f993b41-9a8d-496a-aecc-8bfd6744e0d3 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:38.922664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb5430-c1de-4e57-b21e-6b21590fe374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:38.923809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb5430-c1de-4e57-b21e-6b21590fe374 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:38.924721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb5430-c1de-4e57-b21e-6b21590fe374 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:38.925151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb5430-c1de-4e57-b21e-6b21590fe374 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:38.940636 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb5430-c1de-4e57-b21e-6b21590fe374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-2eeb5430-c1de-4e57-b21e-6b21590fe374 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:38.949240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:38.950099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:38.950834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:38.950834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:38.950834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:38.950834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:38.950834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:38.950834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:38.950834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:38.950834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:38.950834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:38.950834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:38.950834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:22:38.950834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:38.950834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 62/122] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:22:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:38.951475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb5430-c1de-4e57-b21e-6b21590fe374 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:38.987751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc1d58-313b-4fd3-aacc-03e6edcffbb4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:39.026657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f1708-7c93-4429-9d37-2d3f047e665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:39.027284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1708-7c93-4429-9d37-2d3f047e665a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:39.027622 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1708-7c93-4429-9d37-2d3f047e665a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:39.028027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f1708-7c93-4429-9d37-2d3f047e665a None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:39.028601 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f1708-7c93-4429-9d37-2d3f047e665a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:39.028942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f1708-7c93-4429-9d37-2d3f047e665a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:39.030355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f1708-7c93-4429-9d37-2d3f047e665a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:39.041802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 61/123] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:39.042801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f1708-7c93-4429-9d37-2d3f047e665a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:39.061520 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:39.074597 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:39.094322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb5430-c1de-4e57-b21e-6b21590fe374 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:39.142645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496954d-bedc-49aa-99da-6fbd440d3589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:39.144348 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496954d-bedc-49aa-99da-6fbd440d3589 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:39.145130 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496954d-bedc-49aa-99da-6fbd440d3589 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:39.145576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496954d-bedc-49aa-99da-6fbd440d3589 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:39.160054 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f1708-7c93-4429-9d37-2d3f047e665a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:39.160308 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496954d-bedc-49aa-99da-6fbd440d3589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-b496954d-bedc-49aa-99da-6fbd440d3589 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:39.166604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:39.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:39.168247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:39.168247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:39.168247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:39.168247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:39.168247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:39.168247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:39.168247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:39.168247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:39.168247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:39.168247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:39.168247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:39.168247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:39.174484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 63/124] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:39.174484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496954d-bedc-49aa-99da-6fbd440d3589 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:39.209077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b246a7-6832-41f1-aedf-1e1b91aa9cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:39.209890 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b246a7-6832-41f1-aedf-1e1b91aa9cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:39.210265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b246a7-6832-41f1-aedf-1e1b91aa9cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:39.210631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b246a7-6832-41f1-aedf-1e1b91aa9cd6 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:39.213061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b246a7-6832-41f1-aedf-1e1b91aa9cd6 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:39.214634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b246a7-6832-41f1-aedf-1e1b91aa9cd6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:39.216668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 62/125] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:39 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:39.217348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b246a7-6832-41f1-aedf-1e1b91aa9cd6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:39.284667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496954d-bedc-49aa-99da-6fbd440d3589 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:39.321170 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bbc05-2f59-4751-a55e-35c5305909aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:39.322012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bbc05-2f59-4751-a55e-35c5305909aa None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:39.322528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bbc05-2f59-4751-a55e-35c5305909aa None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:39.322936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bbc05-2f59-4751-a55e-35c5305909aa None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:39.323861 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bbc05-2f59-4751-a55e-35c5305909aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:39.324333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bbc05-2f59-4751-a55e-35c5305909aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:39.325912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bbc05-2f59-4751-a55e-35c5305909aa None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:39.335660 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 64/126] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:39.336889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bbc05-2f59-4751-a55e-35c5305909aa None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:39.338346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:39.342673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:39.353296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b246a7-6832-41f1-aedf-1e1b91aa9cd6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:39.390004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb1d69-7817-4b4d-a275-ab2e4527ce42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:39.391014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb1d69-7817-4b4d-a275-ab2e4527ce42 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:39.391492 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb1d69-7817-4b4d-a275-ab2e4527ce42 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:39.392077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb1d69-7817-4b4d-a275-ab2e4527ce42 None admin] PATH_INFO: `/v3/users/82b69c02f3fd4ddc81686f03143f0651` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:39.393857 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb1d69-7817-4b4d-a275-ab2e4527ce42 None admin] RBAC: Authorizing `identity:get_user(user_id=82b69c02f3fd4ddc81686f03143f0651)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:39.395879 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb1d69-7817-4b4d-a275-ab2e4527ce42 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:39.398928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 63/127] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:39 2022] GET /identity/v3/users/82b69c02f3fd4ddc81686f03143f0651 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:39.400553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb1d69-7817-4b4d-a275-ab2e4527ce42 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:39.465219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bbc05-2f59-4751-a55e-35c5305909aa None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:39.511829 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be37599-2c66-4676-9696-2a8d82bd1b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:39.512859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be37599-2c66-4676-9696-2a8d82bd1b8b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:39.513116 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb1d69-7817-4b4d-a275-ab2e4527ce42 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:39.513544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be37599-2c66-4676-9696-2a8d82bd1b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:39.514030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be37599-2c66-4676-9696-2a8d82bd1b8b None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:39.515161 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be37599-2c66-4676-9696-2a8d82bd1b8b None admin] RBAC: Adding query filter params (name=member) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:39.516300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be37599-2c66-4676-9696-2a8d82bd1b8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:39.517744 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be37599-2c66-4676-9696-2a8d82bd1b8b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:39.529768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 65/128] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:39.531683 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be37599-2c66-4676-9696-2a8d82bd1b8b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:39.552776 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56897b-de61-4cdc-b65d-357b49d5507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:39.553456 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56897b-de61-4cdc-b65d-357b49d5507f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:39.553746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56897b-de61-4cdc-b65d-357b49d5507f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:39.554231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56897b-de61-4cdc-b65d-357b49d5507f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:39.554790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56897b-de61-4cdc-b65d-357b49d5507f None admin] RBAC: Adding query filter params (role.id=16095b51965d46fe87bf7fc60e42bdb0, scope.project.id=ce0bf520fc554803bd196782f428d532, user.id=6e9ec43d2aed457589dcc8a3620749d5) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:39.555101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56897b-de61-4cdc-b65d-357b49d5507f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:39.556829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56897b-de61-4cdc-b65d-357b49d5507f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:39.563820 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 64/129] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:39 2022] GET /identity/v3/role_assignments?user.id=6e9ec43d2aed457589dcc8a3620749d5&scope.project.id=ce0bf520fc554803bd196782f428d532&role.id=16095b51965d46fe87bf7fc60e42bdb0 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:39.565220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56897b-de61-4cdc-b65d-357b49d5507f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:39.606466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:39.640348 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:39.702641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be37599-2c66-4676-9696-2a8d82bd1b8b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:39.710693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56897b-de61-4cdc-b65d-357b49d5507f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:39.756151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700a2f7-edd3-435b-9353-9d9bb97589a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:39.756741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700a2f7-edd3-435b-9353-9d9bb97589a6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:39.757302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700a2f7-edd3-435b-9353-9d9bb97589a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:39.757554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700a2f7-edd3-435b-9353-9d9bb97589a6 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:39.760885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700a2f7-edd3-435b-9353-9d9bb97589a6 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:39.761279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700a2f7-edd3-435b-9353-9d9bb97589a6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:39.763004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 65/130] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:39 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:39.763793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700a2f7-edd3-435b-9353-9d9bb97589a6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:39.771436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ce7a9-5cb1-4c8b-8aa5-6b62565f32de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:39.772133 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ce7a9-5cb1-4c8b-8aa5-6b62565f32de None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:39.772416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ce7a9-5cb1-4c8b-8aa5-6b62565f32de None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:39.772714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ce7a9-5cb1-4c8b-8aa5-6b62565f32de None admin] PATH_INFO: `/v3/users/ab56ac1625ce49dfba98e1bb018da72d` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:39.773900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ce7a9-5cb1-4c8b-8aa5-6b62565f32de None admin] RBAC: Authorizing `identity:get_user(user_id=ab56ac1625ce49dfba98e1bb018da72d)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:39.775545 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ce7a9-5cb1-4c8b-8aa5-6b62565f32de None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:39.778613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 66/131] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:39 2022] GET /identity/v3/users/ab56ac1625ce49dfba98e1bb018da72d => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:39.779389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ce7a9-5cb1-4c8b-8aa5-6b62565f32de None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:39.882917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:39.890316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700a2f7-edd3-435b-9353-9d9bb97589a6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:39.943974 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee95162-0cd7-4d52-b977-dab8918ca276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:39.945176 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee95162-0cd7-4d52-b977-dab8918ca276 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:39.947449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ce7a9-5cb1-4c8b-8aa5-6b62565f32de None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:39.948047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee95162-0cd7-4d52-b977-dab8918ca276 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:39.948519 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee95162-0cd7-4d52-b977-dab8918ca276 None admin] PATH_INFO: `/v3/users/ab56ac1625ce49dfba98e1bb018da72d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:39.951292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee95162-0cd7-4d52-b977-dab8918ca276 None admin] RBAC: Authorizing `identity:get_user(user_id=ab56ac1625ce49dfba98e1bb018da72d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:39.953014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee95162-0cd7-4d52-b977-dab8918ca276 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:39.956619 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 66/132] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:39 2022] GET /identity/v3/users/ab56ac1625ce49dfba98e1bb018da72d => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:39.957671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee95162-0cd7-4d52-b977-dab8918ca276 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:40.020020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbe742-a49c-46bf-b77a-129b077d31b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:40.021046 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbe742-a49c-46bf-b77a-129b077d31b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:40.023224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbe742-a49c-46bf-b77a-129b077d31b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:40.023744 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbe742-a49c-46bf-b77a-129b077d31b8 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532/users/82b69c02f3fd4ddc81686f03143f0651/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:40.028795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbe742-a49c-46bf-b77a-129b077d31b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce0bf520fc554803bd196782f428d532, user_id=82b69c02f3fd4ddc81686f03143f0651, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:40.032153 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbe742-a49c-46bf-b77a-129b077d31b8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:40.034889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:40.074193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 67/133] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:22:39 2022] PUT /identity/v3/projects/ce0bf520fc554803bd196782f428d532/users/82b69c02f3fd4ddc81686f03143f0651/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:22:40.075465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbe742-a49c-46bf-b77a-129b077d31b8 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:40.123777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee95162-0cd7-4d52-b977-dab8918ca276 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:40.187690 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930db0e-517e-4ad0-a712-e12ba93a8f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:40.188750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930db0e-517e-4ad0-a712-e12ba93a8f02 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:40.189224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930db0e-517e-4ad0-a712-e12ba93a8f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:40.189686 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930db0e-517e-4ad0-a712-e12ba93a8f02 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:40.192162 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930db0e-517e-4ad0-a712-e12ba93a8f02 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:40.194721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930db0e-517e-4ad0-a712-e12ba93a8f02 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:40.204802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 67/134] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:39 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:40.207820 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004a6b4-46a2-4fe8-b4b2-2907c3c5f8d5 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:40.210745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004a6b4-46a2-4fe8-b4b2-2907c3c5f8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:40.211314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004a6b4-46a2-4fe8-b4b2-2907c3c5f8d5 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:40.213269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 68/135] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:40.215435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004a6b4-46a2-4fe8-b4b2-2907c3c5f8d5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:40.262626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:40.289913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbe742-a49c-46bf-b77a-129b077d31b8 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:40.330428 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26da371-ad9f-4fe2-80d0-35ddd630c561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:40.331456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26da371-ad9f-4fe2-80d0-35ddd630c561 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:40.332012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26da371-ad9f-4fe2-80d0-35ddd630c561 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:40.332477 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26da371-ad9f-4fe2-80d0-35ddd630c561 None admin] PATH_INFO: `/v3/projects/161024fa80304870ad5a26ea4c23b96b` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:40.333865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26da371-ad9f-4fe2-80d0-35ddd630c561 None admin] RBAC: Authorizing `identity:get_project(project_id=161024fa80304870ad5a26ea4c23b96b)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:40.335575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26da371-ad9f-4fe2-80d0-35ddd630c561 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:40.337831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 68/136] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:40 2022] GET /identity/v3/projects/161024fa80304870ad5a26ea4c23b96b => generated 312 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:40.339226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda041d-fe7f-4f6a-8949-9b9e2f98a4aa None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:40.339735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda041d-fe7f-4f6a-8949-9b9e2f98a4aa None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:40.340221 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda041d-fe7f-4f6a-8949-9b9e2f98a4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:40.347085 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004a6b4-46a2-4fe8-b4b2-2907c3c5f8d5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:40.361364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-2eda041d-fe7f-4f6a-8949-9b9e2f98a4aa None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:40.362960 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2eda041d-fe7f-4f6a-8949-9b9e2f98a4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:40.389522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2982a-9f5f-4e30-b1ba-d569bfa3ebca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:40.390237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2982a-9f5f-4e30-b1ba-d569bfa3ebca None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:40.390511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2982a-9f5f-4e30-b1ba-d569bfa3ebca None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:40.390785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2982a-9f5f-4e30-b1ba-d569bfa3ebca None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532/users/ab56ac1625ce49dfba98e1bb018da72d/roles/be3ffa4db9954414868f116f97f3a929` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:40.393909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2982a-9f5f-4e30-b1ba-d569bfa3ebca None admin] RBAC: Authorizing `identity:create_grant(project_id=ce0bf520fc554803bd196782f428d532, user_id=ab56ac1625ce49dfba98e1bb018da72d, role_id=be3ffa4db9954414868f116f97f3a929)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:40.395636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2982a-9f5f-4e30-b1ba-d569bfa3ebca None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:40.420005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 69/137] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:22:40 2022] PUT /identity/v3/projects/ce0bf520fc554803bd196782f428d532/users/ab56ac1625ce49dfba98e1bb018da72d/roles/be3ffa4db9954414868f116f97f3a929 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:22:40.420005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2982a-9f5f-4e30-b1ba-d569bfa3ebca None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:40.455712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:40.471254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 69/138] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:40.605006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:40.615170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2982a-9f5f-4e30-b1ba-d569bfa3ebca None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:40.662200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e27b92-2ad1-4c4d-b4eb-b909502f2bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:40.662885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e27b92-2ad1-4c4d-b4eb-b909502f2bbc None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:40.663171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e27b92-2ad1-4c4d-b4eb-b909502f2bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:40.663445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e27b92-2ad1-4c4d-b4eb-b909502f2bbc None admin] PATH_INFO: `/v3/projects/161024fa80304870ad5a26ea4c23b96b/users/ab56ac1625ce49dfba98e1bb018da72d/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:40.665500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e27b92-2ad1-4c4d-b4eb-b909502f2bbc None admin] RBAC: Authorizing `identity:create_grant(project_id=161024fa80304870ad5a26ea4c23b96b, user_id=ab56ac1625ce49dfba98e1bb018da72d, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:40.667189 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e27b92-2ad1-4c4d-b4eb-b909502f2bbc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:40.669983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec4d37-b0c8-46c7-9a67-c03c79b56b79 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:40.670268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec4d37-b0c8-46c7-9a67-c03c79b56b79 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:40.670541 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec4d37-b0c8-46c7-9a67-c03c79b56b79 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:40.672139 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 70/139] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:40.677769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3180a-fa13-48ad-83e3-2e51f7b806b8 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:40.678065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3180a-fa13-48ad-83e3-2e51f7b806b8 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:40.678338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3180a-fa13-48ad-83e3-2e51f7b806b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:40.692622 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 70/140] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:22:40 2022] PUT /identity/v3/projects/161024fa80304870ad5a26ea4c23b96b/users/ab56ac1625ce49dfba98e1bb018da72d/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:22:40.702024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-b0f3180a-fa13-48ad-83e3-2e51f7b806b8 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:40.703290 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b0f3180a-fa13-48ad-83e3-2e51f7b806b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:40.829672 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 71/141] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:40.933155 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e27b92-2ad1-4c4d-b4eb-b909502f2bbc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:41.076380 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e27b92-2ad1-4c4d-b4eb-b909502f2bbc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:41.121530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772efdf0-6946-4b36-8da6-008820582975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:41.122194 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772efdf0-6946-4b36-8da6-008820582975 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:41.122482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772efdf0-6946-4b36-8da6-008820582975 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:41.122757 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772efdf0-6946-4b36-8da6-008820582975 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:41.133644 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772efdf0-6946-4b36-8da6-008820582975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-772efdf0-6946-4b36-8da6-008820582975 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:41.139281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:41.140163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:41.140927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:41.140927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:41.140927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:41.140927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:41.140927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:41.140927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:41.140927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:41.140927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:41.140927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:41.140927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:41.140927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:41.140927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:41.140927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 71/142] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:41.146510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3180a-fa13-48ad-83e3-2e51f7b806b8 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:41.316034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3180a-fa13-48ad-83e3-2e51f7b806b8 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:41.342447 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:41.354823 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892f1ae-cc0f-4c3b-bd54-10186249e2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:41.355445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892f1ae-cc0f-4c3b-bd54-10186249e2af None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:41.355715 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892f1ae-cc0f-4c3b-bd54-10186249e2af None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:41.356009 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892f1ae-cc0f-4c3b-bd54-10186249e2af None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:41.356684 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892f1ae-cc0f-4c3b-bd54-10186249e2af None admin] RBAC: Adding query filter params (name=member) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:41.356977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892f1ae-cc0f-4c3b-bd54-10186249e2af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:41.358387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892f1ae-cc0f-4c3b-bd54-10186249e2af None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:41.367421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 72/143] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:41.373251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892f1ae-cc0f-4c3b-bd54-10186249e2af None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:41.373480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772efdf0-6946-4b36-8da6-008820582975 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:41.400440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:41.542661 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892f1ae-cc0f-4c3b-bd54-10186249e2af None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:41.565272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772efdf0-6946-4b36-8da6-008820582975 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:41.624784 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a32864-72af-4df0-8320-1818ceb5cac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:41.625364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a32864-72af-4df0-8320-1818ceb5cac2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:41.625640 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a32864-72af-4df0-8320-1818ceb5cac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:41.625915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a32864-72af-4df0-8320-1818ceb5cac2 None admin] PATH_INFO: `/v3/users/ab56ac1625ce49dfba98e1bb018da72d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:41.627105 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a32864-72af-4df0-8320-1818ceb5cac2 None admin] RBAC: Authorizing `identity:get_user(user_id=ab56ac1625ce49dfba98e1bb018da72d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:41.628613 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a32864-72af-4df0-8320-1818ceb5cac2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:41.630805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 72/144] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:41 2022] GET /identity/v3/users/ab56ac1625ce49dfba98e1bb018da72d => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:41.635500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b0d45-135b-46df-8250-53a540f663b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:41.636385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a32864-72af-4df0-8320-1818ceb5cac2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:41.636605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b0d45-135b-46df-8250-53a540f663b3 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:41.637085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b0d45-135b-46df-8250-53a540f663b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:41.637555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b0d45-135b-46df-8250-53a540f663b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:41.644691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b0d45-135b-46df-8250-53a540f663b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:41.646194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b0d45-135b-46df-8250-53a540f663b3 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:41.647863 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 73/145] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:41.653636 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b0d45-135b-46df-8250-53a540f663b3 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:41.744284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:41.841975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:41.873991 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b0d45-135b-46df-8250-53a540f663b3 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:41.892418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a32864-72af-4df0-8320-1818ceb5cac2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:41.982164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a363b4-e587-4191-80ba-fddc0ecbc1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:41.982806 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a363b4-e587-4191-80ba-fddc0ecbc1ea None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:41.983089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a363b4-e587-4191-80ba-fddc0ecbc1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:41.983374 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a363b4-e587-4191-80ba-fddc0ecbc1ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:42.037604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125b143-9299-40ba-9380-f3c8ac6ca0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:42.038287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125b143-9299-40ba-9380-f3c8ac6ca0e9 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:42.038568 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125b143-9299-40ba-9380-f3c8ac6ca0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:42.038833 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125b143-9299-40ba-9380-f3c8ac6ca0e9 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:42.039550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125b143-9299-40ba-9380-f3c8ac6ca0e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:42.041101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125b143-9299-40ba-9380-f3c8ac6ca0e9 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:42.113041 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a363b4-e587-4191-80ba-fddc0ecbc1ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:42.114591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a363b4-e587-4191-80ba-fddc0ecbc1ea None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:42.126324 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 74/146] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:22:41 2022] POST /identity/v3/users => generated 283 bytes in 473 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:22:42.261731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:42.294217 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 73/147] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:22:42.300901 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125b143-9299-40ba-9380-f3c8ac6ca0e9 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:42.303341 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281d003-bfbe-4d35-89e1-ae4ad4904338 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:42.303632 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281d003-bfbe-4d35-89e1-ae4ad4904338 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:42.304052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281d003-bfbe-4d35-89e1-ae4ad4904338 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:42.305460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 74/148] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:22:42.312985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d1a07-9126-4a11-aa5b-5aec08cf5cf9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:42.313581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d1a07-9126-4a11-aa5b-5aec08cf5cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:42.314048 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d1a07-9126-4a11-aa5b-5aec08cf5cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:42.336521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-916d1a07-9126-4a11-aa5b-5aec08cf5cf9 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:42.339437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-916d1a07-9126-4a11-aa5b-5aec08cf5cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:42.536254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 75/149] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:22:42.566676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125b143-9299-40ba-9380-f3c8ac6ca0e9 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:42.590251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:42.620799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6be1b-9c6a-4ec6-9f2e-2257c09e46c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:42.621809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6be1b-9c6a-4ec6-9f2e-2257c09e46c4 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:42.622311 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6be1b-9c6a-4ec6-9f2e-2257c09e46c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:42.622771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6be1b-9c6a-4ec6-9f2e-2257c09e46c4 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:42.624225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6be1b-9c6a-4ec6-9f2e-2257c09e46c4 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:42.626176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6be1b-9c6a-4ec6-9f2e-2257c09e46c4 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:42.628214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 75/150] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:42 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:42.637091 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d1a07-9126-4a11-aa5b-5aec08cf5cf9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:42.807119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d1a07-9126-4a11-aa5b-5aec08cf5cf9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:42.822046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:42.856442 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fffe9f-fe3c-45a4-98aa-3aa40a46c8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:42.857074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fffe9f-fe3c-45a4-98aa-3aa40a46c8ce None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:42.857384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fffe9f-fe3c-45a4-98aa-3aa40a46c8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:42.857654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fffe9f-fe3c-45a4-98aa-3aa40a46c8ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:42.858389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fffe9f-fe3c-45a4-98aa-3aa40a46c8ce None admin] RBAC: Adding query filter params (role.id=3100f6054f7d4d7b984fe19a4d8ba291, scope.project.id=ce0bf520fc554803bd196782f428d532, user.id=ab56ac1625ce49dfba98e1bb018da72d) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:42.858693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fffe9f-fe3c-45a4-98aa-3aa40a46c8ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:42.860164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fffe9f-fe3c-45a4-98aa-3aa40a46c8ce None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:42.873475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 76/151] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:42 2022] GET /identity/v3/role_assignments?user.id=ab56ac1625ce49dfba98e1bb018da72d&scope.project.id=ce0bf520fc554803bd196782f428d532&role.id=3100f6054f7d4d7b984fe19a4d8ba291 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:42.978499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4a3af-c7a4-4bb6-b759-4c21b54c5573 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:42.978850 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4a3af-c7a4-4bb6-b759-4c21b54c5573 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:42.979066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4a3af-c7a4-4bb6-b759-4c21b54c5573 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:42.980862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 76/152] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:42.986588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81658a-2df7-40d2-b19b-267dc6957d99 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:42.986874 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81658a-2df7-40d2-b19b-267dc6957d99 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:42.987148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81658a-2df7-40d2-b19b-267dc6957d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:42.993058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4a3af-c7a4-4bb6-b759-4c21b54c5573 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:43.020705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8d81658a-2df7-40d2-b19b-267dc6957d99 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:43.021950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8d81658a-2df7-40d2-b19b-267dc6957d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:43.137613 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 77/153] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:43.146751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4a3af-c7a4-4bb6-b759-4c21b54c5573 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:43.186552 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a4403-c435-49c0-ab24-35927fc35067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:43.187201 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a4403-c435-49c0-ab24-35927fc35067 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:43.187477 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a4403-c435-49c0-ab24-35927fc35067 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:43.187741 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a4403-c435-49c0-ab24-35927fc35067 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:43.202371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a4403-c435-49c0-ab24-35927fc35067 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-0c7a4403-c435-49c0-ab24-35927fc35067 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:43.208061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:43.208887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:43.209624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:43.209624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:43.209624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:43.209624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:43.209624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:43.209624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:43.209624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:43.209624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:43.209624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:43.209624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:43.209624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:22:43.209624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:43.209624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 77/154] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:43.217115 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81658a-2df7-40d2-b19b-267dc6957d99 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:43.222113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9c23c-b4f3-46d3-8082-92a12090851c None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:43.222398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9c23c-b4f3-46d3-8082-92a12090851c None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:43.222671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9c23c-b4f3-46d3-8082-92a12090851c None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:43.224179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 78/155] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:43.229893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989faa6-51b8-4f42-8e88-24a3433a0fdf None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:43.230170 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989faa6-51b8-4f42-8e88-24a3433a0fdf None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:43.230444 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989faa6-51b8-4f42-8e88-24a3433a0fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:43.252423 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-5989faa6-51b8-4f42-8e88-24a3433a0fdf None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:43.253757 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5989faa6-51b8-4f42-8e88-24a3433a0fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:43.304237 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:43.380753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 79/156] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:22:43.389520 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81658a-2df7-40d2-b19b-267dc6957d99 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:43.392644 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:43.434496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb29c0-3f4c-4080-9655-28e3e6b6fd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:43.435119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb29c0-3f4c-4080-9655-28e3e6b6fd20 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:43.435398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb29c0-3f4c-4080-9655-28e3e6b6fd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:43.435665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb29c0-3f4c-4080-9655-28e3e6b6fd20 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:43.436360 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb29c0-3f4c-4080-9655-28e3e6b6fd20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:43.436655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb29c0-3f4c-4080-9655-28e3e6b6fd20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:43.438042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb29c0-3f4c-4080-9655-28e3e6b6fd20 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:43.451581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 78/157] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:43.458027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989faa6-51b8-4f42-8e88-24a3433a0fdf None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:43.523026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb29c0-3f4c-4080-9655-28e3e6b6fd20 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:43.596912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989faa6-51b8-4f42-8e88-24a3433a0fdf None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:43.665695 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0fc69-782c-4ba9-8c45-0eb41fc52a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:43.666162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0fc69-782c-4ba9-8c45-0eb41fc52a49 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:43.666435 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0fc69-782c-4ba9-8c45-0eb41fc52a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:43.666794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0fc69-782c-4ba9-8c45-0eb41fc52a49 None admin] PATH_INFO: `/v3/users/6e9ec43d2aed457589dcc8a3620749d5` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:43.668143 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0fc69-782c-4ba9-8c45-0eb41fc52a49 None admin] RBAC: Authorizing `identity:get_user(user_id=6e9ec43d2aed457589dcc8a3620749d5)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:43.669765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0fc69-782c-4ba9-8c45-0eb41fc52a49 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:43.671795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 80/158] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:43 2022] GET /identity/v3/users/6e9ec43d2aed457589dcc8a3620749d5 => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:43.677761 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0fc69-782c-4ba9-8c45-0eb41fc52a49 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:43.707058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb29c0-3f4c-4080-9655-28e3e6b6fd20 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:43.740467 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:43.786526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e9e0a-8165-4f1d-9566-5adecccffde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:43.787976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e9e0a-8165-4f1d-9566-5adecccffde6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:43.788668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e9e0a-8165-4f1d-9566-5adecccffde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:43.789084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e9e0a-8165-4f1d-9566-5adecccffde6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:43.813015 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e9e0a-8165-4f1d-9566-5adecccffde6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-253e9e0a-8165-4f1d-9566-5adecccffde6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:43.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:43.821500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:43.822426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:43.822426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:43.822426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:43.822426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:43.822426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:43.822426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:43.822426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:43.822426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:43.822426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:43.822426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:43.822426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:43.822426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:43.825012 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 79/159] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:43.826416 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5aeab-adb7-4c92-955a-3e93a4b95d7e None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:43.826829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5aeab-adb7-4c92-955a-3e93a4b95d7e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:43.827226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5aeab-adb7-4c92-955a-3e93a4b95d7e None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:43.831629 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 80/160] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:43.832972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5aeab-adb7-4c92-955a-3e93a4b95d7e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:43.844406 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0fc69-782c-4ba9-8c45-0eb41fc52a49 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:43.871063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:43.888974 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd9cab-8e6b-4c2d-9bc2-c5255d35b993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:43.889685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd9cab-8e6b-4c2d-9bc2-c5255d35b993 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:43.889962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd9cab-8e6b-4c2d-9bc2-c5255d35b993 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:43.890232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd9cab-8e6b-4c2d-9bc2-c5255d35b993 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:43.891423 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd9cab-8e6b-4c2d-9bc2-c5255d35b993 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:43.893524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd9cab-8e6b-4c2d-9bc2-c5255d35b993 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:43.899986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 81/161] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:43 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:43.901746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd9cab-8e6b-4c2d-9bc2-c5255d35b993 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:43.973220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5aeab-adb7-4c92-955a-3e93a4b95d7e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:44.013956 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd9cab-8e6b-4c2d-9bc2-c5255d35b993 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:44.015622 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:44.021164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532851c-74b2-4c67-af48-a6a5e9af1812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:44.022103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532851c-74b2-4c67-af48-a6a5e9af1812 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:44.022643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532851c-74b2-4c67-af48-a6a5e9af1812 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:44.023121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532851c-74b2-4c67-af48-a6a5e9af1812 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:44.038359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6532851c-74b2-4c67-af48-a6a5e9af1812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-6532851c-74b2-4c67-af48-a6a5e9af1812 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:44.045944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:44.046843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:44.047661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:44.047661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:44.047661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:44.047661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:44.047661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:44.047661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:44.047661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:44.047661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:44.047661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:44.047661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:44.047661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:22:44.047661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:44.049206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 81/162] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:22:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:44.054044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb034c-0c67-4ab1-acaf-65e86fb55b06 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:44.054044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb034c-0c67-4ab1-acaf-65e86fb55b06 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:44.054044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb034c-0c67-4ab1-acaf-65e86fb55b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:44.073321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d4a8b-ebad-4abd-9165-0e394b6bc5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:44.073569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-42cb034c-0c67-4ab1-acaf-65e86fb55b06 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:44.073961 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4a8b-ebad-4abd-9165-0e394b6bc5ed None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:44.074241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4a8b-ebad-4abd-9165-0e394b6bc5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:44.074513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4a8b-ebad-4abd-9165-0e394b6bc5ed None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:44.074709 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-42cb034c-0c67-4ab1-acaf-65e86fb55b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:44.075190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d4a8b-ebad-4abd-9165-0e394b6bc5ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:44.075491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d4a8b-ebad-4abd-9165-0e394b6bc5ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:44.076921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d4a8b-ebad-4abd-9165-0e394b6bc5ed None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:44.102160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 82/163] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:44.103234 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d4a8b-ebad-4abd-9165-0e394b6bc5ed None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:44.162021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 82/164] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:44.163463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb034c-0c67-4ab1-acaf-65e86fb55b06 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:44.195396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:44.297248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d4a8b-ebad-4abd-9165-0e394b6bc5ed None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:44.340089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb034c-0c67-4ab1-acaf-65e86fb55b06 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:44.364166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05e568-5dfe-4673-bab6-83feda07dbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:44.364844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05e568-5dfe-4673-bab6-83feda07dbcb None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:44.365111 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05e568-5dfe-4673-bab6-83feda07dbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:44.365379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05e568-5dfe-4673-bab6-83feda07dbcb None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532/users/6e9ec43d2aed457589dcc8a3620749d5/roles/16095b51965d46fe87bf7fc60e42bdb0` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:44.369414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05e568-5dfe-4673-bab6-83feda07dbcb None admin] RBAC: Authorizing `identity:create_grant(project_id=ce0bf520fc554803bd196782f428d532, user_id=6e9ec43d2aed457589dcc8a3620749d5, role_id=16095b51965d46fe87bf7fc60e42bdb0)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:44.370894 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05e568-5dfe-4673-bab6-83feda07dbcb None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:44.393585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb81048-738d-4342-a33c-3c24a14eb7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:44.394210 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb81048-738d-4342-a33c-3c24a14eb7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:44.394485 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb81048-738d-4342-a33c-3c24a14eb7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:44.394782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb81048-738d-4342-a33c-3c24a14eb7e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:44.396359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb81048-738d-4342-a33c-3c24a14eb7e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:44.396942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb81048-738d-4342-a33c-3c24a14eb7e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:44.399190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb81048-738d-4342-a33c-3c24a14eb7e5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:44.411111 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 83/165] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:22:44 2022] PUT /identity/v3/projects/ce0bf520fc554803bd196782f428d532/users/6e9ec43d2aed457589dcc8a3620749d5/roles/16095b51965d46fe87bf7fc60e42bdb0 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:22:44.412162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05e568-5dfe-4673-bab6-83feda07dbcb None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:44.420915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 83/166] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:44.430267 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb81048-738d-4342-a33c-3c24a14eb7e5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:44.460907 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:44.624829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:44.631512 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05e568-5dfe-4673-bab6-83feda07dbcb None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:44.679015 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb81048-738d-4342-a33c-3c24a14eb7e5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:44.696616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38908e-9b60-4512-bb77-e5dc130a343f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:44.697589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38908e-9b60-4512-bb77-e5dc130a343f None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:44.698007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38908e-9b60-4512-bb77-e5dc130a343f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:44.698418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38908e-9b60-4512-bb77-e5dc130a343f None admin] PATH_INFO: `/v3/users/82b69c02f3fd4ddc81686f03143f0651` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:44.700236 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38908e-9b60-4512-bb77-e5dc130a343f None admin] RBAC: Authorizing `identity:get_user(user_id=82b69c02f3fd4ddc81686f03143f0651)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:44.702600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38908e-9b60-4512-bb77-e5dc130a343f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:44.713440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 84/167] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:44 2022] GET /identity/v3/users/82b69c02f3fd4ddc81686f03143f0651 => generated 247 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:44.714738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38908e-9b60-4512-bb77-e5dc130a343f None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:44.789286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d851049-a54c-46de-8f9d-2959a0feca42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:44.790010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d851049-a54c-46de-8f9d-2959a0feca42 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:44.790355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d851049-a54c-46de-8f9d-2959a0feca42 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:44.790830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d851049-a54c-46de-8f9d-2959a0feca42 None admin] PATH_INFO: `/v3/users/ab56ac1625ce49dfba98e1bb018da72d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:44.793276 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d851049-a54c-46de-8f9d-2959a0feca42 None admin] RBAC: Authorizing `identity:get_user(user_id=ab56ac1625ce49dfba98e1bb018da72d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:44.794751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d851049-a54c-46de-8f9d-2959a0feca42 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:44.796426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 84/168] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:44 2022] GET /identity/v3/users/ab56ac1625ce49dfba98e1bb018da72d => generated 275 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:44.802807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d851049-a54c-46de-8f9d-2959a0feca42 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:44.921327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:44.937340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38908e-9b60-4512-bb77-e5dc130a343f None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:45.001573 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a677eda-9e7a-478e-9eea-d8f9584e76fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:45.002230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a677eda-9e7a-478e-9eea-d8f9584e76fe None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:45.002514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a677eda-9e7a-478e-9eea-d8f9584e76fe None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:45.002787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a677eda-9e7a-478e-9eea-d8f9584e76fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:45.024578 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a677eda-9e7a-478e-9eea-d8f9584e76fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-6a677eda-9e7a-478e-9eea-d8f9584e76fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:45.030687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:45.031565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:45.032391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:45.032391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:45.032391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:45.032391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:45.032391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:45.032391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:45.032391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:45.032391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:45.032391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:45.032391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:45.032391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:45.032391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:45.034764 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 85/169] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:45.035880 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a677eda-9e7a-478e-9eea-d8f9584e76fe None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:45.051266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d851049-a54c-46de-8f9d-2959a0feca42 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:45.117844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277bcc3-fa97-4aac-abf9-80e0fa2365d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:45.120013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277bcc3-fa97-4aac-abf9-80e0fa2365d1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:45.121344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277bcc3-fa97-4aac-abf9-80e0fa2365d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:45.121808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277bcc3-fa97-4aac-abf9-80e0fa2365d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:45.154994 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:45.222899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a677eda-9e7a-478e-9eea-d8f9584e76fe None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:45.254515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277bcc3-fa97-4aac-abf9-80e0fa2365d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:45.256512 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277bcc3-fa97-4aac-abf9-80e0fa2365d1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:45.288987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84607354-a0b0-4877-898d-7c7ed44d1bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:45.290901 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84607354-a0b0-4877-898d-7c7ed44d1bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:45.292174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84607354-a0b0-4877-898d-7c7ed44d1bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:45.292657 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84607354-a0b0-4877-898d-7c7ed44d1bd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:45.334458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:45.422063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 85/170] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:45.424354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277bcc3-fa97-4aac-abf9-80e0fa2365d1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:45.425557 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84607354-a0b0-4877-898d-7c7ed44d1bd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:45.427299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84607354-a0b0-4877-898d-7c7ed44d1bd4 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:45.520386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:45.616594 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277bcc3-fa97-4aac-abf9-80e0fa2365d1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:45.622274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 86/171] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:45.623141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84607354-a0b0-4877-898d-7c7ed44d1bd4 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:45.682203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79b0e7-8f3b-4197-8dbd-cd18e6bdd2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:45.682850 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79b0e7-8f3b-4197-8dbd-cd18e6bdd2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:45.683131 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79b0e7-8f3b-4197-8dbd-cd18e6bdd2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:45.683401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79b0e7-8f3b-4197-8dbd-cd18e6bdd2f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:45.684092 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79b0e7-8f3b-4197-8dbd-cd18e6bdd2f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:45.684394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79b0e7-8f3b-4197-8dbd-cd18e6bdd2f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:45.685785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79b0e7-8f3b-4197-8dbd-cd18e6bdd2f7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:45.695471 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 86/172] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:45.696437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79b0e7-8f3b-4197-8dbd-cd18e6bdd2f7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:45.731924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:45.817858 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84607354-a0b0-4877-898d-7c7ed44d1bd4 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:45.818631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79b0e7-8f3b-4197-8dbd-cd18e6bdd2f7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:45.829307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:45.853104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cc67c-5b97-456d-9524-a325574f6db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:45.853751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cc67c-5b97-456d-9524-a325574f6db8 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:45.854024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cc67c-5b97-456d-9524-a325574f6db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:45.854288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cc67c-5b97-456d-9524-a325574f6db8 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:45.855424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4cc67c-5b97-456d-9524-a325574f6db8 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:45.856940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4cc67c-5b97-456d-9524-a325574f6db8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:45.858892 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 87/173] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:45 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:45.859688 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cc67c-5b97-456d-9524-a325574f6db8 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:45.861442 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b7a3b-b796-432c-9a87-a80a09a2260e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:45.862407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b7a3b-b796-432c-9a87-a80a09a2260e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:45.862828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b7a3b-b796-432c-9a87-a80a09a2260e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:45.863325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b7a3b-b796-432c-9a87-a80a09a2260e None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:45.864943 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b7a3b-b796-432c-9a87-a80a09a2260e None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:45.866553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b7a3b-b796-432c-9a87-a80a09a2260e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:45.868635 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 87/174] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:45 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:45.869568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b7a3b-b796-432c-9a87-a80a09a2260e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:45.959813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cc67c-5b97-456d-9524-a325574f6db8 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:45.980054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:45.983023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b7a3b-b796-432c-9a87-a80a09a2260e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:45.994367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be42f-aa64-4d33-b6e7-46180c94e63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:45.995400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be42f-aa64-4d33-b6e7-46180c94e63a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:45.995916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be42f-aa64-4d33-b6e7-46180c94e63a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:45.996460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be42f-aa64-4d33-b6e7-46180c94e63a None admin] PATH_INFO: `/v3/users/ab56ac1625ce49dfba98e1bb018da72d` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:45.998023 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716be42f-aa64-4d33-b6e7-46180c94e63a None admin] RBAC: Authorizing `identity:get_user(user_id=ab56ac1625ce49dfba98e1bb018da72d)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:45.999582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716be42f-aa64-4d33-b6e7-46180c94e63a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:46.001387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 88/175] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:45 2022] GET /identity/v3/users/ab56ac1625ce49dfba98e1bb018da72d => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:46.002116 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be42f-aa64-4d33-b6e7-46180c94e63a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:46.025230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefdb2b-f7dc-49ef-9517-586d9fd88453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:46.026149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefdb2b-f7dc-49ef-9517-586d9fd88453 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:46.026562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefdb2b-f7dc-49ef-9517-586d9fd88453 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:46.026990 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefdb2b-f7dc-49ef-9517-586d9fd88453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:46.028106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefdb2b-f7dc-49ef-9517-586d9fd88453 None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=ce0bf520fc554803bd196782f428d532, user.id=82b69c02f3fd4ddc81686f03143f0651) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:46.028591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefdb2b-f7dc-49ef-9517-586d9fd88453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:46.030334 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefdb2b-f7dc-49ef-9517-586d9fd88453 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:46.037579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 88/176] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:45 2022] GET /identity/v3/role_assignments?user.id=82b69c02f3fd4ddc81686f03143f0651&scope.project.id=ce0bf520fc554803bd196782f428d532&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:46.039059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8b12-2183-489c-a116-35e1d9a26d51 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:46.039528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8b12-2183-489c-a116-35e1d9a26d51 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:46.040035 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8b12-2183-489c-a116-35e1d9a26d51 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:46.041495 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 89/177] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:46.042433 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b8b12-2183-489c-a116-35e1d9a26d51 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:46.094279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:46.108801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be42f-aa64-4d33-b6e7-46180c94e63a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:46.144547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf9fb7-e0b0-4b38-b950-69c23c2aded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:46.145560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf9fb7-e0b0-4b38-b950-69c23c2aded1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:46.145560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf9fb7-e0b0-4b38-b950-69c23c2aded1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:46.145749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf9fb7-e0b0-4b38-b950-69c23c2aded1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:46.146402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf9fb7-e0b0-4b38-b950-69c23c2aded1 None admin] RBAC: Adding query filter params (role.id=be3ffa4db9954414868f116f97f3a929, scope.project.id=ce0bf520fc554803bd196782f428d532, user.id=ab56ac1625ce49dfba98e1bb018da72d) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:46.146703 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf9fb7-e0b0-4b38-b950-69c23c2aded1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:46.148216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf9fb7-e0b0-4b38-b950-69c23c2aded1 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:46.155002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 89/178] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:46 2022] GET /identity/v3/role_assignments?user.id=ab56ac1625ce49dfba98e1bb018da72d&scope.project.id=ce0bf520fc554803bd196782f428d532&role.id=be3ffa4db9954414868f116f97f3a929 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:46.156272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39b706-2e55-45a3-9949-c8119acaa48d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:46.156529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39b706-2e55-45a3-9949-c8119acaa48d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:46.156796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39b706-2e55-45a3-9949-c8119acaa48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:46.170436 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b8b12-2183-489c-a116-35e1d9a26d51 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:46.176258 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6d39b706-2e55-45a3-9949-c8119acaa48d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:46.177472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6d39b706-2e55-45a3-9949-c8119acaa48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:46.221486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907814cf-82c7-4b0e-becd-72bc900c6026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:46.222440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907814cf-82c7-4b0e-becd-72bc900c6026 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:46.227735 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907814cf-82c7-4b0e-becd-72bc900c6026 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:46.228285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907814cf-82c7-4b0e-becd-72bc900c6026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:46.233870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:46.252013 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 90/179] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:46.255857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac41b07-07f0-4925-9e91-96c28b03eecf None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:46.255857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac41b07-07f0-4925-9e91-96c28b03eecf None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:46.255857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac41b07-07f0-4925-9e91-96c28b03eecf None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:46.255857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 91/180] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:46.260912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b85da-c5ab-4c3c-aa67-cbd11e69ef01 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:46.261186 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b85da-c5ab-4c3c-aa67-cbd11e69ef01 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:46.261456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b85da-c5ab-4c3c-aa67-cbd11e69ef01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:46.285080 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-911b85da-c5ab-4c3c-aa67-cbd11e69ef01 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:46.287626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-911b85da-c5ab-4c3c-aa67-cbd11e69ef01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:46.346390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907814cf-82c7-4b0e-becd-72bc900c6026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:46.347808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907814cf-82c7-4b0e-becd-72bc900c6026 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:46.422321 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:46.518515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 92/181] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:46.542254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 90/182] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:46.548683 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b85da-c5ab-4c3c-aa67-cbd11e69ef01 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:46.652944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907814cf-82c7-4b0e-becd-72bc900c6026 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:46.673341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b85da-c5ab-4c3c-aa67-cbd11e69ef01 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:46.692272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:46.714192 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a33270-acf8-46d4-a868-0b8968e144e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:46.714836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a33270-acf8-46d4-a868-0b8968e144e5 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:46.715109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a33270-acf8-46d4-a868-0b8968e144e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:46.715378 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a33270-acf8-46d4-a868-0b8968e144e5 None admin] PATH_INFO: `/v3/projects/161024fa80304870ad5a26ea4c23b96b` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:46.716554 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a33270-acf8-46d4-a868-0b8968e144e5 None admin] RBAC: Authorizing `identity:get_project(project_id=161024fa80304870ad5a26ea4c23b96b)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:46.718019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a33270-acf8-46d4-a868-0b8968e144e5 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:46.719680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 93/183] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:46 2022] GET /identity/v3/projects/161024fa80304870ad5a26ea4c23b96b => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:46.729140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a33270-acf8-46d4-a868-0b8968e144e5 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:46.839252 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907814cf-82c7-4b0e-becd-72bc900c6026 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:46.885517 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea1daf-804f-4c76-89f1-c7efe6bac34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:46.886219 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea1daf-804f-4c76-89f1-c7efe6bac34f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:46.886498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea1daf-804f-4c76-89f1-c7efe6bac34f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:46.886761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea1daf-804f-4c76-89f1-c7efe6bac34f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:46.896527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:46.899786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea1daf-804f-4c76-89f1-c7efe6bac34f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-bdea1daf-804f-4c76-89f1-c7efe6bac34f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:46.905297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:46.906221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:46.907005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:46.907005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:46.907005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:46.907005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:46.907005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:46.907005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:46.907005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:46.907005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:46.907005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:46.907005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:46.907005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:46.907005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:46.907005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 91/184] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:46.908471 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691708f3-16e2-4ed3-871f-d5d1ec70c8e1 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:46.908471 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691708f3-16e2-4ed3-871f-d5d1ec70c8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:46.908738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691708f3-16e2-4ed3-871f-d5d1ec70c8e1 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:46.910196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 92/185] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:46.917429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691708f3-16e2-4ed3-871f-d5d1ec70c8e1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:46.933070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a33270-acf8-46d4-a868-0b8968e144e5 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:46.984202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07357b-582e-4609-8317-45b7d7e0b60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:46.984955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07357b-582e-4609-8317-45b7d7e0b60e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:46.985385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07357b-582e-4609-8317-45b7d7e0b60e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:46.985916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07357b-582e-4609-8317-45b7d7e0b60e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:46.987116 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07357b-582e-4609-8317-45b7d7e0b60e None admin] RBAC: Adding query filter params (role.id=3100f6054f7d4d7b984fe19a4d8ba291, scope.project.id=161024fa80304870ad5a26ea4c23b96b, user.id=ab56ac1625ce49dfba98e1bb018da72d) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:46.987431 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07357b-582e-4609-8317-45b7d7e0b60e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:46.988931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa07357b-582e-4609-8317-45b7d7e0b60e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:46.996417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 94/186] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:46 2022] GET /identity/v3/role_assignments?user.id=ab56ac1625ce49dfba98e1bb018da72d&scope.project.id=161024fa80304870ad5a26ea4c23b96b&role.id=3100f6054f7d4d7b984fe19a4d8ba291 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:46.999507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ed52a-936d-4446-becf-4862298f4a5e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.001194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ed52a-936d-4446-becf-4862298f4a5e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.001479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ed52a-936d-4446-becf-4862298f4a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.024442 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-770ed52a-936d-4446-becf-4862298f4a5e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:47.025724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-770ed52a-936d-4446-becf-4862298f4a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:47.072326 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691708f3-16e2-4ed3-871f-d5d1ec70c8e1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:47.088253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:47.123989 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccc1fc-8150-4629-9a23-764d5201e224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:47.124649 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccc1fc-8150-4629-9a23-764d5201e224 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.124925 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccc1fc-8150-4629-9a23-764d5201e224 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.125217 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccc1fc-8150-4629-9a23-764d5201e224 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.126152 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccc1fc-8150-4629-9a23-764d5201e224 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:47.126455 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccc1fc-8150-4629-9a23-764d5201e224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:47.128419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccc1fc-8150-4629-9a23-764d5201e224 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:47.134100 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 95/187] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:47.134100 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ed52a-936d-4446-becf-4862298f4a5e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:47.158494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 93/188] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:47.161123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccc1fc-8150-4629-9a23-764d5201e224 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:47.259851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:47.301257 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ed52a-936d-4446-becf-4862298f4a5e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:47.321485 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccc1fc-8150-4629-9a23-764d5201e224 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:47.350309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78475ae1-7fff-470f-9ffc-d6d0943d4400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:47.351492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78475ae1-7fff-470f-9ffc-d6d0943d4400 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.352863 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78475ae1-7fff-470f-9ffc-d6d0943d4400 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.353303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78475ae1-7fff-470f-9ffc-d6d0943d4400 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.371394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78475ae1-7fff-470f-9ffc-d6d0943d4400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:47.376509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb8eb0-e61f-4717-9579-b30f4923daae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:47.377383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb8eb0-e61f-4717-9579-b30f4923daae None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-78475ae1-7fff-470f-9ffc-d6d0943d4400 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:47.377616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:47.378487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:47.379254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:47.379254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:47.379254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:47.379254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:47.379254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:47.379254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:47.379254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:47.379254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:47.379254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:47.379254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:47.379254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:22:47.379254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:47.379254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 96/189] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:22:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:47.380558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb8eb0-e61f-4717-9579-b30f4923daae None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.380981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb8eb0-e61f-4717-9579-b30f4923daae None admin] PATH_INFO: `/v3/users/91357caa348a417683b2d20722c8368e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.385595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78475ae1-7fff-470f-9ffc-d6d0943d4400 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:47.400102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb8eb0-e61f-4717-9579-b30f4923daae None admin] RBAC: Authorizing `identity:get_user(user_id=91357caa348a417683b2d20722c8368e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:47.401941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb8eb0-e61f-4717-9579-b30f4923daae None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:47.404021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 94/190] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:47 2022] GET /identity/v3/users/91357caa348a417683b2d20722c8368e => generated 283 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:47.411714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb8eb0-e61f-4717-9579-b30f4923daae None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:47.440900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:47.525544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78475ae1-7fff-470f-9ffc-d6d0943d4400 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:47.538426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb8eb0-e61f-4717-9579-b30f4923daae None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:47.562439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:47.570294 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba495f7-f6ba-4c1e-8c1a-3b52e82880cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:47.570922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba495f7-f6ba-4c1e-8c1a-3b52e82880cb None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.571217 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba495f7-f6ba-4c1e-8c1a-3b52e82880cb None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.571483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba495f7-f6ba-4c1e-8c1a-3b52e82880cb None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.572213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba495f7-f6ba-4c1e-8c1a-3b52e82880cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:47.572512 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba495f7-f6ba-4c1e-8c1a-3b52e82880cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:47.573928 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba495f7-f6ba-4c1e-8c1a-3b52e82880cb None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:47.580963 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3eb35b-fd29-421f-a8f8-a5ad4b3a8894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:47.581589 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3eb35b-fd29-421f-a8f8-a5ad4b3a8894 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.581848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3eb35b-fd29-421f-a8f8-a5ad4b3a8894 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.582127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3eb35b-fd29-421f-a8f8-a5ad4b3a8894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.588491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 97/191] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:47.588491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba495f7-f6ba-4c1e-8c1a-3b52e82880cb None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:47.668006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3eb35b-fd29-421f-a8f8-a5ad4b3a8894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:47.670086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3eb35b-fd29-421f-a8f8-a5ad4b3a8894 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:47.728450 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:47.734964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba495f7-f6ba-4c1e-8c1a-3b52e82880cb None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:47.769212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb75fa-f0d2-4c3e-91d0-e6c5dbaed7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:47.769852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb75fa-f0d2-4c3e-91d0-e6c5dbaed7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.770125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb75fa-f0d2-4c3e-91d0-e6c5dbaed7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.770388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb75fa-f0d2-4c3e-91d0-e6c5dbaed7d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.771502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb75fa-f0d2-4c3e-91d0-e6c5dbaed7d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:47.772986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb75fa-f0d2-4c3e-91d0-e6c5dbaed7d1 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:47.774765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 98/192] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:47.775518 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb75fa-f0d2-4c3e-91d0-e6c5dbaed7d1 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:47.795543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 95/193] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:22:47.798074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb5fd6-0634-4e53-9e57-65f03661a53b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.798364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb5fd6-0634-4e53-9e57-65f03661a53b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.798636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb5fd6-0634-4e53-9e57-65f03661a53b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.800075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 96/194] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:47.800643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb5fd6-0634-4e53-9e57-65f03661a53b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:47.822714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:47.881137 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb75fa-f0d2-4c3e-91d0-e6c5dbaed7d1 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:47.903368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb5fd6-0634-4e53-9e57-65f03661a53b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:47.920268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d654240-95a8-4cd3-8dbf-dfa8cbe663bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:47.920831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d654240-95a8-4cd3-8dbf-dfa8cbe663bd None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.921111 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d654240-95a8-4cd3-8dbf-dfa8cbe663bd None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.921380 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d654240-95a8-4cd3-8dbf-dfa8cbe663bd None admin] PATH_INFO: `/v3/users/91357caa348a417683b2d20722c8368e` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.922517 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d654240-95a8-4cd3-8dbf-dfa8cbe663bd None admin] RBAC: Authorizing `identity:get_user(user_id=91357caa348a417683b2d20722c8368e)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:47.924105 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d654240-95a8-4cd3-8dbf-dfa8cbe663bd None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:47.925715 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 99/195] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:47 2022] GET /identity/v3/users/91357caa348a417683b2d20722c8368e => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:47.926479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d654240-95a8-4cd3-8dbf-dfa8cbe663bd None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:47.939982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11158c2-1ea6-4d9f-ac97-af820cfdd505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:47.940609 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11158c2-1ea6-4d9f-ac97-af820cfdd505 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.940920 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11158c2-1ea6-4d9f-ac97-af820cfdd505 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.941187 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11158c2-1ea6-4d9f-ac97-af820cfdd505 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.941847 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11158c2-1ea6-4d9f-ac97-af820cfdd505 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:47.943302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11158c2-1ea6-4d9f-ac97-af820cfdd505 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:47.961531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:47.966732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 97/196] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:22:47 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:47.968099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6ae13-2225-49e9-8496-1f6489cfb52f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:47.968383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6ae13-2225-49e9-8496-1f6489cfb52f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:47.968652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6ae13-2225-49e9-8496-1f6489cfb52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:47.994205 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f4b6ae13-2225-49e9-8496-1f6489cfb52f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:47.996173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f4b6ae13-2225-49e9-8496-1f6489cfb52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:48.042519 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d654240-95a8-4cd3-8dbf-dfa8cbe663bd None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:48.076966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 98/197] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:48.077871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6ae13-2225-49e9-8496-1f6489cfb52f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:48.090005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:48.093151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620bf89-6133-454b-ae22-aea537f7300e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:48.094422 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620bf89-6133-454b-ae22-aea537f7300e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.094920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620bf89-6133-454b-ae22-aea537f7300e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.095856 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620bf89-6133-454b-ae22-aea537f7300e None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.097623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620bf89-6133-454b-ae22-aea537f7300e None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:48.099991 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620bf89-6133-454b-ae22-aea537f7300e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:48.101998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 100/198] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:47 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:48.109891 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620bf89-6133-454b-ae22-aea537f7300e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:48.206012 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6ae13-2225-49e9-8496-1f6489cfb52f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:48.230669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:48.234920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620bf89-6133-454b-ae22-aea537f7300e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:48.249995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1716233-ef99-439a-99b8-1a1b11cc16f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:48.251149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1716233-ef99-439a-99b8-1a1b11cc16f8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.251596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1716233-ef99-439a-99b8-1a1b11cc16f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.252062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1716233-ef99-439a-99b8-1a1b11cc16f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.278028 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa79e87-12a9-412f-9dc9-52d94e84a357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:48.278750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa79e87-12a9-412f-9dc9-52d94e84a357 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.279098 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa79e87-12a9-412f-9dc9-52d94e84a357 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.279370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa79e87-12a9-412f-9dc9-52d94e84a357 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.280146 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa79e87-12a9-412f-9dc9-52d94e84a357 None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6, user.id=91357caa348a417683b2d20722c8368e) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:48.280454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa79e87-12a9-412f-9dc9-52d94e84a357 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:48.282073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa79e87-12a9-412f-9dc9-52d94e84a357 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:48.289159 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 101/199] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:48 2022] GET /identity/v3/role_assignments?user.id=91357caa348a417683b2d20722c8368e&scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:48.337417 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1716233-ef99-439a-99b8-1a1b11cc16f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:48.339001 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1716233-ef99-439a-99b8-1a1b11cc16f8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:48.348907 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:48.380566 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa79e87-12a9-412f-9dc9-52d94e84a357 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:48.479118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 99/200] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:48.486006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1716233-ef99-439a-99b8-1a1b11cc16f8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:48.508741 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa79e87-12a9-412f-9dc9-52d94e84a357 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:48.546857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00508257-74f1-4f8e-af38-462927347b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:48.547542 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508257-74f1-4f8e-af38-462927347b9a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.547819 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508257-74f1-4f8e-af38-462927347b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.548121 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508257-74f1-4f8e-af38-462927347b9a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.560311 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00508257-74f1-4f8e-af38-462927347b9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-00508257-74f1-4f8e-af38-462927347b9a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:48.566254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:48.567334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:48.568269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:48.568269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:48.568269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:48.568269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:48.568269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:48.568269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:48.568269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:48.568269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:48.568269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:48.568269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:48.568269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:22:48.568269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:48.568269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 102/201] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:48.573923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00508257-74f1-4f8e-af38-462927347b9a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:48.610157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:48.622238 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1716233-ef99-439a-99b8-1a1b11cc16f8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:48.633823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:48.687489 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bdecf-4afc-4d9b-bb72-e3c496e7d9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:48.688173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bdecf-4afc-4d9b-bb72-e3c496e7d9aa None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.688447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bdecf-4afc-4d9b-bb72-e3c496e7d9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.688714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bdecf-4afc-4d9b-bb72-e3c496e7d9aa None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.690765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bdecf-4afc-4d9b-bb72-e3c496e7d9aa None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:48.692098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bdecf-4afc-4d9b-bb72-e3c496e7d9aa None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:48.693970 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 100/202] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:48 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:48.700714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bdecf-4afc-4d9b-bb72-e3c496e7d9aa None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:48.769309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00508257-74f1-4f8e-af38-462927347b9a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:48.802490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bdecf-4afc-4d9b-bb72-e3c496e7d9aa None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:48.832489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f8b38-70d9-4fe0-ba48-6aef7dcfa3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:48.833116 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f8b38-70d9-4fe0-ba48-6aef7dcfa3aa None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.833389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f8b38-70d9-4fe0-ba48-6aef7dcfa3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.833656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f8b38-70d9-4fe0-ba48-6aef7dcfa3aa None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.834325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f8b38-70d9-4fe0-ba48-6aef7dcfa3aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:48.834620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f8b38-70d9-4fe0-ba48-6aef7dcfa3aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:48.836028 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f8b38-70d9-4fe0-ba48-6aef7dcfa3aa None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:48.837915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e11391-43b4-4ba4-8746-6c5255f533c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:48.838537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11391-43b4-4ba4-8746-6c5255f533c8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:48.838812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11391-43b4-4ba4-8746-6c5255f533c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:48.839077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11391-43b4-4ba4-8746-6c5255f533c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:48.839801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e11391-43b4-4ba4-8746-6c5255f533c8 None admin] RBAC: Adding query filter params (role.id=be3ffa4db9954414868f116f97f3a929, scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6, user.id=91357caa348a417683b2d20722c8368e) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:48.840140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e11391-43b4-4ba4-8746-6c5255f533c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:48.841556 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e11391-43b4-4ba4-8746-6c5255f533c8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:48.847570 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 101/203] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:48 2022] GET /identity/v3/role_assignments?user.id=91357caa348a417683b2d20722c8368e&scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6&role.id=be3ffa4db9954414868f116f97f3a929 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:48.851472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 103/204] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:48.858477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e11391-43b4-4ba4-8746-6c5255f533c8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:48.926837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:48.997134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e11391-43b4-4ba4-8746-6c5255f533c8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:49.048039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f1203-18ac-4195-8032-99a1033aff01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:49.048621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f1203-18ac-4195-8032-99a1033aff01 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:49.048901 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f1203-18ac-4195-8032-99a1033aff01 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:49.049172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f1203-18ac-4195-8032-99a1033aff01 None admin] PATH_INFO: `/v3/users/6e9ec43d2aed457589dcc8a3620749d5` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:49.057066 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f1203-18ac-4195-8032-99a1033aff01 None admin] RBAC: Authorizing `identity:get_user(user_id=6e9ec43d2aed457589dcc8a3620749d5)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:49.058945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f1203-18ac-4195-8032-99a1033aff01 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:49.060831 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 102/205] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:48 2022] GET /identity/v3/users/6e9ec43d2aed457589dcc8a3620749d5 => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:49.067829 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f8b38-70d9-4fe0-ba48-6aef7dcfa3aa None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:49.191100 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:49.208264 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f8b38-70d9-4fe0-ba48-6aef7dcfa3aa None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:49.254384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34428550-6f41-4198-b234-5072467c505e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:49.255387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34428550-6f41-4198-b234-5072467c505e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:49.255852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34428550-6f41-4198-b234-5072467c505e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:49.256333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34428550-6f41-4198-b234-5072467c505e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:49.332614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34428550-6f41-4198-b234-5072467c505e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:49.334574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34428550-6f41-4198-b234-5072467c505e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:49.420728 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:49.445905 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 104/206] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:49.453058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f1203-18ac-4195-8032-99a1033aff01 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:49.600572 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f1203-18ac-4195-8032-99a1033aff01 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:49.614423 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:49.661588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920493b6-620a-4f2b-8cf6-77cf10fc18b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:49.662239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920493b6-620a-4f2b-8cf6-77cf10fc18b0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:49.662518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920493b6-620a-4f2b-8cf6-77cf10fc18b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:49.662790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920493b6-620a-4f2b-8cf6-77cf10fc18b0 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:49.663937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920493b6-620a-4f2b-8cf6-77cf10fc18b0 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:49.665484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920493b6-620a-4f2b-8cf6-77cf10fc18b0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:49.667341 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 103/207] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:49 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:49.673592 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34428550-6f41-4198-b234-5072467c505e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:49.826391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34428550-6f41-4198-b234-5072467c505e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:49.874993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0910e-1647-406e-acb2-20bfa84648a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:49.875523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0910e-1647-406e-acb2-20bfa84648a6 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:49.875792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0910e-1647-406e-acb2-20bfa84648a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:49.876081 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0910e-1647-406e-acb2-20bfa84648a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:49.876812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0910e-1647-406e-acb2-20bfa84648a6 None admin] RBAC: Adding query filter params (role.id=16095b51965d46fe87bf7fc60e42bdb0, scope.project.id=ce0bf520fc554803bd196782f428d532, user.id=6e9ec43d2aed457589dcc8a3620749d5) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:49.877112 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0910e-1647-406e-acb2-20bfa84648a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:49.878566 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0910e-1647-406e-acb2-20bfa84648a6 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:49.888487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 105/208] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:49 2022] GET /identity/v3/role_assignments?user.id=6e9ec43d2aed457589dcc8a3620749d5&scope.project.id=ce0bf520fc554803bd196782f428d532&role.id=16095b51965d46fe87bf7fc60e42bdb0 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:50.522427 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e663d-eeab-4e00-933a-ce3aa3a34544 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:50.522941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e663d-eeab-4e00-933a-ce3aa3a34544 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:50.522941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e663d-eeab-4e00-933a-ce3aa3a34544 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:50.524617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 104/209] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:50.530171 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb2b96-0d7c-48e8-8d5e-cebc8aa5a4a8 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:50.530584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb2b96-0d7c-48e8-8d5e-cebc8aa5a4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:50.530757 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb2b96-0d7c-48e8-8d5e-cebc8aa5a4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:50.551074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-02cb2b96-0d7c-48e8-8d5e-cebc8aa5a4a8 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:50.552384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-02cb2b96-0d7c-48e8-8d5e-cebc8aa5a4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:50.578211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38210d9-d9c5-4666-9e82-3e63ea4355f5 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:50.579244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38210d9-d9c5-4666-9e82-3e63ea4355f5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:50.580007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38210d9-d9c5-4666-9e82-3e63ea4355f5 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:50.582388 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 105/210] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:22:50.588179 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce49dbf-5254-4abe-9af4-5d155c8f3906 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:50.588698 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce49dbf-5254-4abe-9af4-5d155c8f3906 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:50.589176 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce49dbf-5254-4abe-9af4-5d155c8f3906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:50.608421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:50.616995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1ce49dbf-5254-4abe-9af4-5d155c8f3906 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:50.618769 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1ce49dbf-5254-4abe-9af4-5d155c8f3906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:50.629964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 106/211] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:50.687291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 106/212] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:50.855531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37945781-0c12-44d7-9cc7-8fc8cd6eae78 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:50.856206 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37945781-0c12-44d7-9cc7-8fc8cd6eae78 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:50.856699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37945781-0c12-44d7-9cc7-8fc8cd6eae78 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:50.858297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 107/213] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:50.863833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3d411-b62d-48ae-83e6-03100f8f4495 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:50.864154 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3d411-b62d-48ae-83e6-03100f8f4495 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:50.864645 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3d411-b62d-48ae-83e6-03100f8f4495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:50.883904 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e5d3d411-b62d-48ae-83e6-03100f8f4495 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:50.885488 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e5d3d411-b62d-48ae-83e6-03100f8f4495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:50.903022 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37945781-0c12-44d7-9cc7-8fc8cd6eae78 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:50.960338 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 107/214] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:50.967157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3d411-b62d-48ae-83e6-03100f8f4495 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.007565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:51.032352 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37945781-0c12-44d7-9cc7-8fc8cd6eae78 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.069189 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95c33d-f7ff-4bfb-8c74-4a46e9973a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.069819 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95c33d-f7ff-4bfb-8c74-4a46e9973a28 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.070087 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95c33d-f7ff-4bfb-8c74-4a46e9973a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.070349 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95c33d-f7ff-4bfb-8c74-4a46e9973a28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.080350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95c33d-f7ff-4bfb-8c74-4a46e9973a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:51.081883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3d411-b62d-48ae-83e6-03100f8f4495 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-8b95c33d-f7ff-4bfb-8c74-4a46e9973a28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:51.085499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:51.086356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:51.087574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:51.087574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:51.087574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:51.087574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:51.087574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:51.087574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:51.087574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:51.087574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:51.087574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:51.087574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:51.087574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:51.087574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:51.087574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 108/215] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:51.088624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f58868-d2bc-4ffe-8aef-38533c572762 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.088624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f58868-d2bc-4ffe-8aef-38533c572762 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.089294 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f58868-d2bc-4ffe-8aef-38533c572762 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.090743 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 109/216] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:51.092948 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f58868-d2bc-4ffe-8aef-38533c572762 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.121115 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71db15f-631f-4faa-a8ec-c5b678ecfced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.122064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71db15f-631f-4faa-a8ec-c5b678ecfced None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.122331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71db15f-631f-4faa-a8ec-c5b678ecfced None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.122596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71db15f-631f-4faa-a8ec-c5b678ecfced None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.134254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71db15f-631f-4faa-a8ec-c5b678ecfced None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e71db15f-631f-4faa-a8ec-c5b678ecfced None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:51.139186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:51.140352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:51.141163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:51.141163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:51.141163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:51.141163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:51.141163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:51.141163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:51.141163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:51.141163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:51.141163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:51.141163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:51.141163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:51.141163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:51.141163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 108/217] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:51.141794 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e9a72-cff4-43ef-a9d1-2a734cdfc579 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.141794 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e9a72-cff4-43ef-a9d1-2a734cdfc579 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.141998 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e9a72-cff4-43ef-a9d1-2a734cdfc579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.148370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:51.160568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f10e9a72-cff4-43ef-a9d1-2a734cdfc579 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:51.161753 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f10e9a72-cff4-43ef-a9d1-2a734cdfc579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:51.214254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f58868-d2bc-4ffe-8aef-38533c572762 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.229257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 109/218] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:51.232926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e9a72-cff4-43ef-a9d1-2a734cdfc579 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.254587 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b47d11-cefc-4e2a-94e8-8b40591c298d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.255206 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b47d11-cefc-4e2a-94e8-8b40591c298d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.255482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b47d11-cefc-4e2a-94e8-8b40591c298d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.255747 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b47d11-cefc-4e2a-94e8-8b40591c298d None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.256451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b47d11-cefc-4e2a-94e8-8b40591c298d None admin] RBAC: Adding query filter params (name=member) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:51.256751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b47d11-cefc-4e2a-94e8-8b40591c298d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:51.258139 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b47d11-cefc-4e2a-94e8-8b40591c298d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:51.264364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:51.267083 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 110/219] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:51.269385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b47d11-cefc-4e2a-94e8-8b40591c298d None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.352080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e9a72-cff4-43ef-a9d1-2a734cdfc579 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.376993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b47d11-cefc-4e2a-94e8-8b40591c298d None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.378417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:51.391283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc0b15-7f40-4f1f-ac45-c168048e9bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.392324 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc0b15-7f40-4f1f-ac45-c168048e9bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.392869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc0b15-7f40-4f1f-ac45-c168048e9bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.393329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc0b15-7f40-4f1f-ac45-c168048e9bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.394186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc0b15-7f40-4f1f-ac45-c168048e9bc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:51.394708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc0b15-7f40-4f1f-ac45-c168048e9bc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:51.396480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc0b15-7f40-4f1f-ac45-c168048e9bc7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:51.410534 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 110/220] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:51.411652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc0b15-7f40-4f1f-ac45-c168048e9bc7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.413372 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f4c02-e082-4c11-bcb5-8d76b7f5088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.414007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f4c02-e082-4c11-bcb5-8d76b7f5088a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.414270 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f4c02-e082-4c11-bcb5-8d76b7f5088a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.414534 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f4c02-e082-4c11-bcb5-8d76b7f5088a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.415759 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f4c02-e082-4c11-bcb5-8d76b7f5088a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:51.417251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f4c02-e082-4c11-bcb5-8d76b7f5088a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:51.418859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 111/221] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:51.419614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f4c02-e082-4c11-bcb5-8d76b7f5088a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.506529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:51.521221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc0b15-7f40-4f1f-ac45-c168048e9bc7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.534938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f4c02-e082-4c11-bcb5-8d76b7f5088a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.554042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706f105-c628-4b4d-8068-c10ecbe1b676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.555077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706f105-c628-4b4d-8068-c10ecbe1b676 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.555352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706f105-c628-4b4d-8068-c10ecbe1b676 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.555856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706f105-c628-4b4d-8068-c10ecbe1b676 None admin] PATH_INFO: `/v3/users/833fbd75dc844396a7ccbe19696f6357` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.567729 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff8942-f188-4ae1-b484-ca77263bbec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.568413 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff8942-f188-4ae1-b484-ca77263bbec2 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.568563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706f105-c628-4b4d-8068-c10ecbe1b676 None admin] RBAC: Authorizing `identity:get_user(user_id=833fbd75dc844396a7ccbe19696f6357)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:51.568677 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff8942-f188-4ae1-b484-ca77263bbec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.568947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff8942-f188-4ae1-b484-ca77263bbec2 None admin] PATH_INFO: `/v3/users/91357caa348a417683b2d20722c8368e` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.570084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706f105-c628-4b4d-8068-c10ecbe1b676 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:51.570188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff8942-f188-4ae1-b484-ca77263bbec2 None admin] RBAC: Authorizing `identity:get_user(user_id=91357caa348a417683b2d20722c8368e)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:51.572030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff8942-f188-4ae1-b484-ca77263bbec2 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:51.572265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 111/222] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:51 2022] GET /identity/v3/users/833fbd75dc844396a7ccbe19696f6357 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:51.572846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706f105-c628-4b4d-8068-c10ecbe1b676 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.573650 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 112/223] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:51 2022] GET /identity/v3/users/91357caa348a417683b2d20722c8368e => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:51.574332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff8942-f188-4ae1-b484-ca77263bbec2 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.617221 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:51.677772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706f105-c628-4b4d-8068-c10ecbe1b676 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.682468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff8942-f188-4ae1-b484-ca77263bbec2 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.713610 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5432d97-f436-4f3a-9f99-739cc5af239f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.714223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5432d97-f436-4f3a-9f99-739cc5af239f None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.714518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5432d97-f436-4f3a-9f99-739cc5af239f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.714765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5432d97-f436-4f3a-9f99-739cc5af239f None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.715441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5432d97-f436-4f3a-9f99-739cc5af239f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:51.716930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5432d97-f436-4f3a-9f99-739cc5af239f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:51.717648 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198059dd-2123-4484-a78c-088e3ece7f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.718291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198059dd-2123-4484-a78c-088e3ece7f8a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.718563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198059dd-2123-4484-a78c-088e3ece7f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.718827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198059dd-2123-4484-a78c-088e3ece7f8a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.729689 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198059dd-2123-4484-a78c-088e3ece7f8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-198059dd-2123-4484-a78c-088e3ece7f8a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:51.736547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:51.737468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:51.738288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:51.738288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:51.738288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:51.738288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:51.738288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:51.738288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:51.738288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:51.738288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:51.738288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:51.738288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:51.738288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:22:51.738288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:51.738288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 113/224] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:22:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:51.739001 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198059dd-2123-4484-a78c-088e3ece7f8a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.740929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:51.752616 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 112/225] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:22:51 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:22:51.752616 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5432d97-f436-4f3a-9f99-739cc5af239f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.860272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:51.864831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198059dd-2123-4484-a78c-088e3ece7f8a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.872043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5432d97-f436-4f3a-9f99-739cc5af239f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:51.912846 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c843b-dbc2-4014-b7af-4f8ed1bf3651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.913253 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a852148e-98a4-4e96-b596-55edd65adb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:51.913480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c843b-dbc2-4014-b7af-4f8ed1bf3651 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.913898 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852148e-98a4-4e96-b596-55edd65adb66 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:51.914171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852148e-98a4-4e96-b596-55edd65adb66 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.914269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c843b-dbc2-4014-b7af-4f8ed1bf3651 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:51.914430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852148e-98a4-4e96-b596-55edd65adb66 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.914530 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c843b-dbc2-4014-b7af-4f8ed1bf3651 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:51.915638 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a852148e-98a4-4e96-b596-55edd65adb66 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:51.917126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a852148e-98a4-4e96-b596-55edd65adb66 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:51.918773 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 113/226] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:51 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:51.919546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a852148e-98a4-4e96-b596-55edd65adb66 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:51.990252 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:51.991078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c843b-dbc2-4014-b7af-4f8ed1bf3651 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:51.992777 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c843b-dbc2-4014-b7af-4f8ed1bf3651 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:52.033696 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a852148e-98a4-4e96-b596-55edd65adb66 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:52.073784 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028b81e-7f04-48f5-b79f-8fb4a643acbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:52.074682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028b81e-7f04-48f5-b79f-8fb4a643acbd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:52.075113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028b81e-7f04-48f5-b79f-8fb4a643acbd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:52.075381 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028b81e-7f04-48f5-b79f-8fb4a643acbd None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:52.076690 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028b81e-7f04-48f5-b79f-8fb4a643acbd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:52.076690 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028b81e-7f04-48f5-b79f-8fb4a643acbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:52.077731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028b81e-7f04-48f5-b79f-8fb4a643acbd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:52.092452 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 114/227] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:52.092452 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028b81e-7f04-48f5-b79f-8fb4a643acbd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:52.120010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:52.125662 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 114/228] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:52.126619 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c843b-dbc2-4014-b7af-4f8ed1bf3651 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:52.211151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028b81e-7f04-48f5-b79f-8fb4a643acbd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:52.243440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:52.252899 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1772c06-ad1e-4147-a4f8-66a682b2b9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:52.254228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1772c06-ad1e-4147-a4f8-66a682b2b9ff None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:52.254592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1772c06-ad1e-4147-a4f8-66a682b2b9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:52.254923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1772c06-ad1e-4147-a4f8-66a682b2b9ff None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6/users/91357caa348a417683b2d20722c8368e/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:52.257425 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1772c06-ad1e-4147-a4f8-66a682b2b9ff None admin] RBAC: Authorizing `identity:create_grant(project_id=7ab355ce83f94db6a7c315ba27ae52a6, user_id=91357caa348a417683b2d20722c8368e, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:52.259970 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1772c06-ad1e-4147-a4f8-66a682b2b9ff None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:52.264454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c843b-dbc2-4014-b7af-4f8ed1bf3651 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:52.279417 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 115/229] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:22:52 2022] PUT /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6/users/91357caa348a417683b2d20722c8368e/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:22:52.280699 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1772c06-ad1e-4147-a4f8-66a682b2b9ff None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:52.306176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce5fe6-fa22-4144-b60f-cb3190578be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:52.307230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce5fe6-fa22-4144-b60f-cb3190578be8 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:52.307686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce5fe6-fa22-4144-b60f-cb3190578be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:52.308231 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce5fe6-fa22-4144-b60f-cb3190578be8 None admin] PATH_INFO: `/v3/users/91357caa348a417683b2d20722c8368e` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:52.309895 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce5fe6-fa22-4144-b60f-cb3190578be8 None admin] RBAC: Authorizing `identity:get_user(user_id=91357caa348a417683b2d20722c8368e)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:52.311721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce5fe6-fa22-4144-b60f-cb3190578be8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:52.314034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 115/230] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:52 2022] GET /identity/v3/users/91357caa348a417683b2d20722c8368e => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:52.320676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce5fe6-fa22-4144-b60f-cb3190578be8 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:52.424905 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:52.451227 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1772c06-ad1e-4147-a4f8-66a682b2b9ff None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:52.485515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce5fe6-fa22-4144-b60f-cb3190578be8 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:52.502106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab53ec-4a45-45ca-bfc1-b61aacc99221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:52.502698 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab53ec-4a45-45ca-bfc1-b61aacc99221 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:52.503128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab53ec-4a45-45ca-bfc1-b61aacc99221 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:52.503403 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab53ec-4a45-45ca-bfc1-b61aacc99221 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:52.505151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab53ec-4a45-45ca-bfc1-b61aacc99221 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:52.507074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ab53ec-4a45-45ca-bfc1-b61aacc99221 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:52.509195 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 116/231] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:52 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:52.516180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab53ec-4a45-45ca-bfc1-b61aacc99221 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:52.535180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbaea8-96bb-4479-a606-b0017d4851d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:52.535834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbaea8-96bb-4479-a606-b0017d4851d6 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:52.536389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbaea8-96bb-4479-a606-b0017d4851d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:52.536658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbaea8-96bb-4479-a606-b0017d4851d6 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:52.537838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbaea8-96bb-4479-a606-b0017d4851d6 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:52.539353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbaea8-96bb-4479-a606-b0017d4851d6 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:52.544869 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 116/232] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:52 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:52.547839 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbaea8-96bb-4479-a606-b0017d4851d6 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:52.618540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:52.659556 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbaea8-96bb-4479-a606-b0017d4851d6 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:52.691978 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ab53ec-4a45-45ca-bfc1-b61aacc99221 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:52.696940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436576f4-57ff-481c-9007-c3e2d4c6e310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:52.697957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436576f4-57ff-481c-9007-c3e2d4c6e310 None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:52.698432 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436576f4-57ff-481c-9007-c3e2d4c6e310 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:52.698907 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436576f4-57ff-481c-9007-c3e2d4c6e310 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6/users/91357caa348a417683b2d20722c8368e/roles/be3ffa4db9954414868f116f97f3a929` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:52.701594 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436576f4-57ff-481c-9007-c3e2d4c6e310 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ab355ce83f94db6a7c315ba27ae52a6, user_id=91357caa348a417683b2d20722c8368e, role_id=be3ffa4db9954414868f116f97f3a929)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:52.703305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436576f4-57ff-481c-9007-c3e2d4c6e310 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:52.706276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:52.728275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 117/233] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:22:52 2022] PUT /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6/users/91357caa348a417683b2d20722c8368e/roles/be3ffa4db9954414868f116f97f3a929 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:22:52.786638 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f56a7-a4fa-40f8-ad94-baaab28818dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:52.787277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f56a7-a4fa-40f8-ad94-baaab28818dc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:52.787553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f56a7-a4fa-40f8-ad94-baaab28818dc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:52.787819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f56a7-a4fa-40f8-ad94-baaab28818dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:52.788582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f56a7-a4fa-40f8-ad94-baaab28818dc None admin] RBAC: Adding query filter params (role.id=3100f6054f7d4d7b984fe19a4d8ba291, scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6, user.id=833fbd75dc844396a7ccbe19696f6357) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:52.788887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f56a7-a4fa-40f8-ad94-baaab28818dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:52.790636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f56a7-a4fa-40f8-ad94-baaab28818dc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:52.804906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 117/234] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:52 2022] GET /identity/v3/role_assignments?user.id=833fbd75dc844396a7ccbe19696f6357&scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6&role.id=3100f6054f7d4d7b984fe19a4d8ba291 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:54.059607 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b66af-1225-4ca1-8d82-15fa7e82a155 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.060165 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b66af-1225-4ca1-8d82-15fa7e82a155 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.060165 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b66af-1225-4ca1-8d82-15fa7e82a155 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.061695 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 118/235] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:54.067176 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e34cd2-7258-488d-aed1-5258bf0852ae None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.067572 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e34cd2-7258-488d-aed1-5258bf0852ae None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.067736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e34cd2-7258-488d-aed1-5258bf0852ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.091730 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-80e34cd2-7258-488d-aed1-5258bf0852ae None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:54.093032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-80e34cd2-7258-488d-aed1-5258bf0852ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:54.166266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 118/236] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:54.263632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d27b2-7e83-476e-ac2b-227f880acdd7 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.264008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d27b2-7e83-476e-ac2b-227f880acdd7 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.264212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d27b2-7e83-476e-ac2b-227f880acdd7 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.265968 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 119/237] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:54.271257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92caac7-5809-4f9f-95d2-9af8dd324e15 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.271553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92caac7-5809-4f9f-95d2-9af8dd324e15 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.271817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92caac7-5809-4f9f-95d2-9af8dd324e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.295751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f92caac7-5809-4f9f-95d2-9af8dd324e15 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:54.297025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f92caac7-5809-4f9f-95d2-9af8dd324e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:54.359407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:54.387936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 119/238] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:54.580448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d27b2-7e83-476e-ac2b-227f880acdd7 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:54.690960 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d27b2-7e83-476e-ac2b-227f880acdd7 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:54.705929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cebec-a0ab-4d94-9c17-36b195afea6c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.706259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cebec-a0ab-4d94-9c17-36b195afea6c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.706501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cebec-a0ab-4d94-9c17-36b195afea6c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.708033 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 120/239] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:22:54.713081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd30176-7682-40dc-9e37-a5365a8fd348 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.713642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd30176-7682-40dc-9e37-a5365a8fd348 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.714107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd30176-7682-40dc-9e37-a5365a8fd348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.733281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e5a33-1539-4377-beff-ac6481e1f149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:54.733947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e5a33-1539-4377-beff-ac6481e1f149 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.734224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e5a33-1539-4377-beff-ac6481e1f149 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.734495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e5a33-1539-4377-beff-ac6481e1f149 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.736583 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e5a33-1539-4377-beff-ac6481e1f149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:54.738081 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e5a33-1539-4377-beff-ac6481e1f149 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:54.740131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 120/240] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:54.745326 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e5a33-1539-4377-beff-ac6481e1f149 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:54.753494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9dd30176-7682-40dc-9e37-a5365a8fd348 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:54.753494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9dd30176-7682-40dc-9e37-a5365a8fd348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:54.812881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:54.843223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 121/241] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:54.844176 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd30176-7682-40dc-9e37-a5365a8fd348 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:54.877316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e5a33-1539-4377-beff-ac6481e1f149 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:54.946740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c8682-a567-43bd-b0ae-db91d3880ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:54.947327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c8682-a567-43bd-b0ae-db91d3880ca3 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:54.947607 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c8682-a567-43bd-b0ae-db91d3880ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:54.947868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c8682-a567-43bd-b0ae-db91d3880ca3 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:54.948553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c8682-a567-43bd-b0ae-db91d3880ca3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:54.950006 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c8682-a567-43bd-b0ae-db91d3880ca3 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:54.994546 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 121/242] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:22:54 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:22:55.030210 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd30176-7682-40dc-9e37-a5365a8fd348 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:55.054661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:55.089927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45359a8-2279-45d8-962a-964b71e04f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:55.090757 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45359a8-2279-45d8-962a-964b71e04f58 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:55.090919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45359a8-2279-45d8-962a-964b71e04f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:55.091197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45359a8-2279-45d8-962a-964b71e04f58 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:55.112527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45359a8-2279-45d8-962a-964b71e04f58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b45359a8-2279-45d8-962a-964b71e04f58 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:55.117569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:55.118585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:55.119379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:55.119379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:55.119379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:55.119379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:55.119379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:55.119379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:55.119379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:55.119379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:55.119379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:55.119379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:55.119379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:22:55.119379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:55.119379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 122/243] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:55.125396 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c8682-a567-43bd-b0ae-db91d3880ca3 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:55.223223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87849c54-1cd7-422a-9e57-62bc18b88b07 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:55.223536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87849c54-1cd7-422a-9e57-62bc18b88b07 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:55.223692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87849c54-1cd7-422a-9e57-62bc18b88b07 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:55.226655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 123/244] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:55.233140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fff9f0-5021-4f3f-b8f0-e3a51c094f42 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:55.233420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fff9f0-5021-4f3f-b8f0-e3a51c094f42 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:55.233692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fff9f0-5021-4f3f-b8f0-e3a51c094f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:55.256533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d0fff9f0-5021-4f3f-b8f0-e3a51c094f42 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:55.257746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d0fff9f0-5021-4f3f-b8f0-e3a51c094f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:55.271799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c8682-a567-43bd-b0ae-db91d3880ca3 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:55.295915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:55.330631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a72d4-af03-4f73-9063-dcfc7d17a09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:55.331843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a72d4-af03-4f73-9063-dcfc7d17a09b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:55.332298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a72d4-af03-4f73-9063-dcfc7d17a09b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:55.332742 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a72d4-af03-4f73-9063-dcfc7d17a09b None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:55.333586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a72d4-af03-4f73-9063-dcfc7d17a09b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:55.334056 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a72d4-af03-4f73-9063-dcfc7d17a09b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:55.335642 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a72d4-af03-4f73-9063-dcfc7d17a09b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:55.344539 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 124/245] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:55.345400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fff9f0-5021-4f3f-b8f0-e3a51c094f42 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:55.349839 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 122/246] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:55.349839 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183e50c-6a6b-4ba7-a902-481014bb1267 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:55.349839 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183e50c-6a6b-4ba7-a902-481014bb1267 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:55.349839 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183e50c-6a6b-4ba7-a902-481014bb1267 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:55.349839 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 123/247] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:55.354931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183e50c-6a6b-4ba7-a902-481014bb1267 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:55.477025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fff9f0-5021-4f3f-b8f0-e3a51c094f42 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:55.478970 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:55.490398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183e50c-6a6b-4ba7-a902-481014bb1267 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:55.528816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ad228-3377-4d6e-b835-b451ec1dbb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:55.529845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493711c-d65d-4a76-ba3d-9fb065b27fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:55.529966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ad228-3377-4d6e-b835-b451ec1dbb93 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:55.530460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ad228-3377-4d6e-b835-b451ec1dbb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:55.530562 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493711c-d65d-4a76-ba3d-9fb065b27fba None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:55.530765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493711c-d65d-4a76-ba3d-9fb065b27fba None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:55.531032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493711c-d65d-4a76-ba3d-9fb065b27fba None admin] PATH_INFO: `/v3/users/c78467d9763446b58050b82e57347140` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:55.531405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ad228-3377-4d6e-b835-b451ec1dbb93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:55.546228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ad228-3377-4d6e-b835-b451ec1dbb93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:55.549028 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493711c-d65d-4a76-ba3d-9fb065b27fba None admin] RBAC: Authorizing `identity:get_user(user_id=c78467d9763446b58050b82e57347140)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:55.550517 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493711c-d65d-4a76-ba3d-9fb065b27fba None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ac2ad228-3377-4d6e-b835-b451ec1dbb93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:55.552400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:55.553320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:55.554150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:55.554150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:55.554150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:55.554150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:55.554150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:55.554150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:55.554150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:55.554150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:55.554150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:55.554150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:55.554150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:22:55.554150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:55.554781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 124/248] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:55 2022] GET /identity/v3/users/c78467d9763446b58050b82e57347140 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:55.554781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d06e71-ac41-4c5c-acc3-82f591e562bf None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:55.554781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d06e71-ac41-4c5c-acc3-82f591e562bf None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:55.554781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d06e71-ac41-4c5c-acc3-82f591e562bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:55.557556 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 125/249] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:22:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:55.566538 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ad228-3377-4d6e-b835-b451ec1dbb93 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:55.585344 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-79d06e71-ac41-4c5c-acc3-82f591e562bf None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:55.586943 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-79d06e71-ac41-4c5c-acc3-82f591e562bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:55.636370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:55.689823 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 125/250] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:55.691178 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d06e71-ac41-4c5c-acc3-82f591e562bf None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:55.706675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ad228-3377-4d6e-b835-b451ec1dbb93 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:55.740835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef9b64-f066-40f4-b870-8f6d0cd46c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:55.741377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef9b64-f066-40f4-b870-8f6d0cd46c0e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:55.741643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef9b64-f066-40f4-b870-8f6d0cd46c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:55.741921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef9b64-f066-40f4-b870-8f6d0cd46c0e None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:55.742570 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef9b64-f066-40f4-b870-8f6d0cd46c0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:55.742916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef9b64-f066-40f4-b870-8f6d0cd46c0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:55.744409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef9b64-f066-40f4-b870-8f6d0cd46c0e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:55.758432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 126/251] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:55.758432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef9b64-f066-40f4-b870-8f6d0cd46c0e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:55.790529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:55.811445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d06e71-ac41-4c5c-acc3-82f591e562bf None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:55.875161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef9b64-f066-40f4-b870-8f6d0cd46c0e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:55.912970 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf55fd-c469-4146-8ff8-77598a55a4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:55.913628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf55fd-c469-4146-8ff8-77598a55a4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:55.913900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf55fd-c469-4146-8ff8-77598a55a4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:55.914169 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf55fd-c469-4146-8ff8-77598a55a4f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:55.925346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf55fd-c469-4146-8ff8-77598a55a4f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-abbf55fd-c469-4146-8ff8-77598a55a4f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:55.930951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:55.931832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:55.932671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:55.932671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:55.932671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:55.932671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:55.932671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:55.932671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:55.932671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:55.932671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:55.932671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:55.932671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:55.932671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:55.932671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:55.932671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 127/252] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:55.933320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76932ac-a35e-4c2a-86b2-4a3ebd92f6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:55.933404 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf55fd-c469-4146-8ff8-77598a55a4f7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:55.933471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76932ac-a35e-4c2a-86b2-4a3ebd92f6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:55.933471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76932ac-a35e-4c2a-86b2-4a3ebd92f6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:55.934158 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76932ac-a35e-4c2a-86b2-4a3ebd92f6e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:55.990143 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:56.031440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76932ac-a35e-4c2a-86b2-4a3ebd92f6e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:56.033788 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76932ac-a35e-4c2a-86b2-4a3ebd92f6e5 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:56.060422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:56.082133 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf55fd-c469-4146-8ff8-77598a55a4f7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:56.126646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b033660-06f2-42cc-917f-bcb691fac8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:56.127713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b033660-06f2-42cc-917f-bcb691fac8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:56.128016 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b033660-06f2-42cc-917f-bcb691fac8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:56.128418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b033660-06f2-42cc-917f-bcb691fac8b3 None admin] PATH_INFO: `/v3/users/91357caa348a417683b2d20722c8368e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:56.129906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b033660-06f2-42cc-917f-bcb691fac8b3 None admin] RBAC: Authorizing `identity:get_user(user_id=91357caa348a417683b2d20722c8368e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:56.131715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b033660-06f2-42cc-917f-bcb691fac8b3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:56.133329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 128/253] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:55 2022] GET /identity/v3/users/91357caa348a417683b2d20722c8368e => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:56.136509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b033660-06f2-42cc-917f-bcb691fac8b3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:56.151915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 126/254] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:56.152929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76932ac-a35e-4c2a-86b2-4a3ebd92f6e5 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:56.224382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:56.264990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76932ac-a35e-4c2a-86b2-4a3ebd92f6e5 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:56.273461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b033660-06f2-42cc-917f-bcb691fac8b3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:56.308085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f87c4a-f947-43e9-bed3-b93fdfd89949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:56.308721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f87c4a-f947-43e9-bed3-b93fdfd89949 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:56.309070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f87c4a-f947-43e9-bed3-b93fdfd89949 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:56.309339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f87c4a-f947-43e9-bed3-b93fdfd89949 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:56.310760 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4a311-24e5-46c9-b01f-1723afbc62e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:56.311568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4a311-24e5-46c9-b01f-1723afbc62e3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:56.312074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4a311-24e5-46c9-b01f-1723afbc62e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:56.312441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4a311-24e5-46c9-b01f-1723afbc62e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:56.313195 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4a311-24e5-46c9-b01f-1723afbc62e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:56.313590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4a311-24e5-46c9-b01f-1723afbc62e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:56.315135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4a311-24e5-46c9-b01f-1723afbc62e3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:56.320325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f87c4a-f947-43e9-bed3-b93fdfd89949 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:56.324649 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 129/255] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:56.325469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4a311-24e5-46c9-b01f-1723afbc62e3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-08f87c4a-f947-43e9-bed3-b93fdfd89949 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:56.328448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:56.331049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:56.332972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:56.332972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:56.332972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:56.332972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:56.332972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:56.332972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:56.332972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:56.332972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:56.332972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:56.332972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:56.332972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:22:56.332972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:22:56.332972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 127/256] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:22:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:56.332972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f87c4a-f947-43e9-bed3-b93fdfd89949 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:56.348931 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:56.442400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4a311-24e5-46c9-b01f-1723afbc62e3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:56.470093 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f87c4a-f947-43e9-bed3-b93fdfd89949 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:56.482224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c75021-b463-4307-b7bf-a12d2c3bdbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:56.482916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c75021-b463-4307-b7bf-a12d2c3bdbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:56.483194 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c75021-b463-4307-b7bf-a12d2c3bdbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:56.483457 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c75021-b463-4307-b7bf-a12d2c3bdbc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:56.502094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:56.511740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d94cb-6c55-4751-8747-4ee87e674395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:56.512418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d94cb-6c55-4751-8747-4ee87e674395 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:56.512698 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d94cb-6c55-4751-8747-4ee87e674395 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:56.512973 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d94cb-6c55-4751-8747-4ee87e674395 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:56.514302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d94cb-6c55-4751-8747-4ee87e674395 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:56.515772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d94cb-6c55-4751-8747-4ee87e674395 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:56.518021 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 128/257] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:56 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:56.518813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d94cb-6c55-4751-8747-4ee87e674395 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:56.560301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c75021-b463-4307-b7bf-a12d2c3bdbc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:56.561670 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c75021-b463-4307-b7bf-a12d2c3bdbc0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:56.576554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:56.631977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d94cb-6c55-4751-8747-4ee87e674395 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:56.668693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 130/258] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:56.669678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c75021-b463-4307-b7bf-a12d2c3bdbc0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:56.674389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cff19-d46f-4e66-b691-49cf6ccd11da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:56.675204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cff19-d46f-4e66-b691-49cf6ccd11da None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:56.675621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cff19-d46f-4e66-b691-49cf6ccd11da None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:56.676157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cff19-d46f-4e66-b691-49cf6ccd11da None admin] PATH_INFO: `/v3/users/833fbd75dc844396a7ccbe19696f6357` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:56.677437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cff19-d46f-4e66-b691-49cf6ccd11da None admin] RBAC: Authorizing `identity:get_user(user_id=833fbd75dc844396a7ccbe19696f6357)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:56.679182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cff19-d46f-4e66-b691-49cf6ccd11da None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:56.681236 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 129/259] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:56 2022] GET /identity/v3/users/833fbd75dc844396a7ccbe19696f6357 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:56.682281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cff19-d46f-4e66-b691-49cf6ccd11da None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:56.767344 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:56.778906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c75021-b463-4307-b7bf-a12d2c3bdbc0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:56.793745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:56.814688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138a0ed-7f6c-45dc-a041-796b2eefb3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:56.815290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138a0ed-7f6c-45dc-a041-796b2eefb3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:56.815567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138a0ed-7f6c-45dc-a041-796b2eefb3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:56.815829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138a0ed-7f6c-45dc-a041-796b2eefb3b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:56.816512 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138a0ed-7f6c-45dc-a041-796b2eefb3b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:56.816808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138a0ed-7f6c-45dc-a041-796b2eefb3b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:56.818161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138a0ed-7f6c-45dc-a041-796b2eefb3b7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:56.823527 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cff19-d46f-4e66-b691-49cf6ccd11da None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:56.826814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 131/260] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:22:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:56.827163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138a0ed-7f6c-45dc-a041-796b2eefb3b7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:56.863312 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac712b6-bceb-4dd6-b1cc-85e08e9c60d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:56.864237 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac712b6-bceb-4dd6-b1cc-85e08e9c60d0 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:56.864655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac712b6-bceb-4dd6-b1cc-85e08e9c60d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:56.865061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac712b6-bceb-4dd6-b1cc-85e08e9c60d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:56.865926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac712b6-bceb-4dd6-b1cc-85e08e9c60d0 None admin] RBAC: Adding query filter params (role.id=16095b51965d46fe87bf7fc60e42bdb0, scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6, user.id=c78467d9763446b58050b82e57347140) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:56.866372 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac712b6-bceb-4dd6-b1cc-85e08e9c60d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:56.868113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac712b6-bceb-4dd6-b1cc-85e08e9c60d0 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:56.880298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 130/261] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:56 2022] GET /identity/v3/role_assignments?user.id=c78467d9763446b58050b82e57347140&scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6&role.id=16095b51965d46fe87bf7fc60e42bdb0 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:56.881516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac712b6-bceb-4dd6-b1cc-85e08e9c60d0 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:56.935950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138a0ed-7f6c-45dc-a041-796b2eefb3b7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:56.968060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0f4e2-ddbe-4494-9f27-b9c9b524b4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:56.968503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0f4e2-ddbe-4494-9f27-b9c9b524b4de None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:56.968776 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0f4e2-ddbe-4494-9f27-b9c9b524b4de None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:56.969042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0f4e2-ddbe-4494-9f27-b9c9b524b4de None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:56.972046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0f4e2-ddbe-4494-9f27-b9c9b524b4de None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:56.973507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0f4e2-ddbe-4494-9f27-b9c9b524b4de None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:56.975219 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 132/262] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:56 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:56.976014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0f4e2-ddbe-4494-9f27-b9c9b524b4de None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:57.088430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:57.094971 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac712b6-bceb-4dd6-b1cc-85e08e9c60d0 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:57.137542 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:57.148326 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c828875-e9c4-4566-a3a6-581d74cd90d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:57.149807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c828875-e9c4-4566-a3a6-581d74cd90d4 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.149807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c828875-e9c4-4566-a3a6-581d74cd90d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.149807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c828875-e9c4-4566-a3a6-581d74cd90d4 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.151001 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c828875-e9c4-4566-a3a6-581d74cd90d4 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:57.153507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c828875-e9c4-4566-a3a6-581d74cd90d4 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:57.156265 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 131/263] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:56 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:57.157047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c828875-e9c4-4566-a3a6-581d74cd90d4 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:57.194095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0f4e2-ddbe-4494-9f27-b9c9b524b4de None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:57.259274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee18f06-c1ea-4804-a156-ee826aa25f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:57.260189 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee18f06-c1ea-4804-a156-ee826aa25f45 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.260471 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee18f06-c1ea-4804-a156-ee826aa25f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.260741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee18f06-c1ea-4804-a156-ee826aa25f45 None admin] PATH_INFO: `/v3/users/91357caa348a417683b2d20722c8368e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.262077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee18f06-c1ea-4804-a156-ee826aa25f45 None admin] RBAC: Authorizing `identity:get_user(user_id=91357caa348a417683b2d20722c8368e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:57.263540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee18f06-c1ea-4804-a156-ee826aa25f45 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:57.265174 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 133/264] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:56 2022] GET /identity/v3/users/91357caa348a417683b2d20722c8368e => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:57.265909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee18f06-c1ea-4804-a156-ee826aa25f45 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:57.352841 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c828875-e9c4-4566-a3a6-581d74cd90d4 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:57.400642 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe312c19-37ec-4de8-b05b-acae7c0733b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:57.401362 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe312c19-37ec-4de8-b05b-acae7c0733b2 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.401645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe312c19-37ec-4de8-b05b-acae7c0733b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.401919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe312c19-37ec-4de8-b05b-acae7c0733b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.402651 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe312c19-37ec-4de8-b05b-acae7c0733b2 None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6, user.id=91357caa348a417683b2d20722c8368e) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:57.402961 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe312c19-37ec-4de8-b05b-acae7c0733b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:57.404693 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe312c19-37ec-4de8-b05b-acae7c0733b2 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:57.411573 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 132/265] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:57 2022] GET /identity/v3/role_assignments?user.id=91357caa348a417683b2d20722c8368e&scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:57.412448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe312c19-37ec-4de8-b05b-acae7c0733b2 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:57.436710 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee18f06-c1ea-4804-a156-ee826aa25f45 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:57.466654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:57.509281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13948051-3653-418c-887b-2c89f52824c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:57.510096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13948051-3653-418c-887b-2c89f52824c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.510387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13948051-3653-418c-887b-2c89f52824c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.510660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13948051-3653-418c-887b-2c89f52824c1 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6/users/833fbd75dc844396a7ccbe19696f6357/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.512960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13948051-3653-418c-887b-2c89f52824c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ab355ce83f94db6a7c315ba27ae52a6, user_id=833fbd75dc844396a7ccbe19696f6357, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:57.514444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13948051-3653-418c-887b-2c89f52824c1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:57.533386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 134/266] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:22:57 2022] PUT /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6/users/833fbd75dc844396a7ccbe19696f6357/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:22:57.535115 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea579cc-86e8-4e0a-b63d-547a2febb55f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.535664 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea579cc-86e8-4e0a-b63d-547a2febb55f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.535960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea579cc-86e8-4e0a-b63d-547a2febb55f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.536928 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:57.539022 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 135/267] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:57.545609 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9818e-5aaf-4093-b419-49db35369201 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.545926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9818e-5aaf-4093-b419-49db35369201 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.546447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9818e-5aaf-4093-b419-49db35369201 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.548352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 136/268] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:57.549559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8aab6-7360-47e2-98d5-cfaad3e0b72b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.549895 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8aab6-7360-47e2-98d5-cfaad3e0b72b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.550171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8aab6-7360-47e2-98d5-cfaad3e0b72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.573516 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f0f8aab6-7360-47e2-98d5-cfaad3e0b72b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:57.574811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f0f8aab6-7360-47e2-98d5-cfaad3e0b72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:57.623167 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe312c19-37ec-4de8-b05b-acae7c0733b2 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:57.686533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 137/269] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:22:57.687042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24997887-26a2-4beb-b9c0-943308e03b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:57.687855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c0d51-6125-4fb6-bca2-6f815e9491ac None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.688181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c0d51-6125-4fb6-bca2-6f815e9491ac None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.688420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24997887-26a2-4beb-b9c0-943308e03b05 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:57.688535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c0d51-6125-4fb6-bca2-6f815e9491ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.689064 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24997887-26a2-4beb-b9c0-943308e03b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:57.689515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24997887-26a2-4beb-b9c0-943308e03b05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:57.707732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-882c0d51-6125-4fb6-bca2-6f815e9491ac None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:57.709299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-882c0d51-6125-4fb6-bca2-6f815e9491ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:57.783514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24997887-26a2-4beb-b9c0-943308e03b05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:57.785602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24997887-26a2-4beb-b9c0-943308e03b05 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:57.792994 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 138/270] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:57.864701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:57.918947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 133/271] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:22:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:22:57.925017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c0d51-6125-4fb6-bca2-6f815e9491ac None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:57.971796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:58.051907 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c0d51-6125-4fb6-bca2-6f815e9491ac None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:58.086323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415e739-59aa-46dd-9a0e-ee1b4db9d824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:58.086986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415e739-59aa-46dd-9a0e-ee1b4db9d824 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:58.087408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415e739-59aa-46dd-9a0e-ee1b4db9d824 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:58.087692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415e739-59aa-46dd-9a0e-ee1b4db9d824 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:58.088838 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415e739-59aa-46dd-9a0e-ee1b4db9d824 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:58.090271 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415e739-59aa-46dd-9a0e-ee1b4db9d824 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:58.092241 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 139/272] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:22:57 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:58.097923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24997887-26a2-4beb-b9c0-943308e03b05 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:58.133866 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415e739-59aa-46dd-9a0e-ee1b4db9d824 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:58.239984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:58.244813 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415e739-59aa-46dd-9a0e-ee1b4db9d824 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:58.261365 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24997887-26a2-4beb-b9c0-943308e03b05 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:58.286977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91043b-32c4-4940-b816-d8c76a3b9a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:58.287990 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91043b-32c4-4940-b816-d8c76a3b9a32 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:58.287990 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91043b-32c4-4940-b816-d8c76a3b9a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:58.288255 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91043b-32c4-4940-b816-d8c76a3b9a32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:58.298682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91043b-32c4-4940-b816-d8c76a3b9a32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:58.303501 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb7477-79f2-47d1-a442-58f3d4375780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-fa91043b-32c4-4940-b816-d8c76a3b9a32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:58.303653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:58.304514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:58.305290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:58.305290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:58.305290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:58.305290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:58.305290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:58.305290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:58.305290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:58.305290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:58.305290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:58.305290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:58.305290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:22:58.305290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:58.305290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 140/273] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:22:58.305988 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91043b-32c4-4940-b816-d8c76a3b9a32 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:58.311605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb7477-79f2-47d1-a442-58f3d4375780 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:58.312225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb7477-79f2-47d1-a442-58f3d4375780 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:58.312666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb7477-79f2-47d1-a442-58f3d4375780 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:58.313543 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb7477-79f2-47d1-a442-58f3d4375780 None admin] RBAC: Adding query filter params (role.id=be3ffa4db9954414868f116f97f3a929, scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6, user.id=91357caa348a417683b2d20722c8368e) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:58.313980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb7477-79f2-47d1-a442-58f3d4375780 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:58.315680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb7477-79f2-47d1-a442-58f3d4375780 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:58.322916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 134/274] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:22:58 2022] GET /identity/v3/role_assignments?user.id=91357caa348a417683b2d20722c8368e&scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6&role.id=be3ffa4db9954414868f116f97f3a929 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:58.323879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb7477-79f2-47d1-a442-58f3d4375780 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:58.353737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:58.494166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb7477-79f2-47d1-a442-58f3d4375780 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:58.513578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91043b-32c4-4940-b816-d8c76a3b9a32 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:58.532458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad32cf3-19b8-4901-b4d5-f24a85baa224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:58.533650 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad32cf3-19b8-4901-b4d5-f24a85baa224 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:58.534090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad32cf3-19b8-4901-b4d5-f24a85baa224 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:58.534490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad32cf3-19b8-4901-b4d5-f24a85baa224 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:58.535452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad32cf3-19b8-4901-b4d5-f24a85baa224 None admin] RBAC: Adding query filter params (name=member) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:58.535933 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad32cf3-19b8-4901-b4d5-f24a85baa224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:58.537468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad32cf3-19b8-4901-b4d5-f24a85baa224 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:58.547120 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 135/275] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:58.550447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a0918-7127-4eaa-8718-960b35164ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:58.551218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a0918-7127-4eaa-8718-960b35164ffc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:58.551593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a0918-7127-4eaa-8718-960b35164ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:58.552224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a0918-7127-4eaa-8718-960b35164ffc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:58.553342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad32cf3-19b8-4901-b4d5-f24a85baa224 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:58.553551 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a0918-7127-4eaa-8718-960b35164ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:58.555267 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a0918-7127-4eaa-8718-960b35164ffc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:58.557630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 141/276] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:22:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:58.563074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a0918-7127-4eaa-8718-960b35164ffc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:58.599419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:58.646346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:58.663368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad32cf3-19b8-4901-b4d5-f24a85baa224 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:58.688980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a0918-7127-4eaa-8718-960b35164ffc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:58.705566 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147c16e-8111-4040-8462-0cc8d4912f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:58.706787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147c16e-8111-4040-8462-0cc8d4912f71 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:58.707199 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147c16e-8111-4040-8462-0cc8d4912f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:58.707598 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147c16e-8111-4040-8462-0cc8d4912f71 None admin] PATH_INFO: `/v3/users/87a3aa28af424890829a6cfd63905975` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:58.728395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147c16e-8111-4040-8462-0cc8d4912f71 None admin] RBAC: Authorizing `identity:get_user(user_id=87a3aa28af424890829a6cfd63905975)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:58.730074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147c16e-8111-4040-8462-0cc8d4912f71 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:58.734712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 136/277] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:22:58 2022] GET /identity/v3/users/87a3aa28af424890829a6cfd63905975 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:22:58.738590 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147c16e-8111-4040-8462-0cc8d4912f71 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:58.754266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadb213-44d5-46e2-9215-4613f255627d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:58.755395 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadb213-44d5-46e2-9215-4613f255627d None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:58.755926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadb213-44d5-46e2-9215-4613f255627d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:58.756383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadb213-44d5-46e2-9215-4613f255627d None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:58.757225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadb213-44d5-46e2-9215-4613f255627d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:58.758964 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadb213-44d5-46e2-9215-4613f255627d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:58.794387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 142/278] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:22:58 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:22:58.889081 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147c16e-8111-4040-8462-0cc8d4912f71 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:58.928331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f86aaf-09b9-4aca-b2ef-e9aecf0e5b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:58.929405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f86aaf-09b9-4aca-b2ef-e9aecf0e5b3f None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:58.929867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f86aaf-09b9-4aca-b2ef-e9aecf0e5b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:58.930329 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f86aaf-09b9-4aca-b2ef-e9aecf0e5b3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:58.931349 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f86aaf-09b9-4aca-b2ef-e9aecf0e5b3f None admin] RBAC: Adding query filter params (role.id=3100f6054f7d4d7b984fe19a4d8ba291, user.id=87a3aa28af424890829a6cfd63905975, scope.system=all) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:22:58.931848 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f86aaf-09b9-4aca-b2ef-e9aecf0e5b3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:22:58.933620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f86aaf-09b9-4aca-b2ef-e9aecf0e5b3f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:22:58.940350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 137/279] 10.222.0.62 () {56 vars in 1308 bytes} [Thu Sep 1 16:22:58 2022] GET /identity/v3/role_assignments?user.id=87a3aa28af424890829a6cfd63905975&scope.system=all&role.id=3100f6054f7d4d7b984fe19a4d8ba291 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:22:59.150580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26252-1935-4959-942c-597001dedba5 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:59.150907 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26252-1935-4959-942c-597001dedba5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:59.151108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26252-1935-4959-942c-597001dedba5 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:59.152867 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 143/280] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:59.158004 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d068fc-8ccc-466f-bd10-bcde02bbab90 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:59.158286 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d068fc-8ccc-466f-bd10-bcde02bbab90 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:59.158563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d068fc-8ccc-466f-bd10-bcde02bbab90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:59.177753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-93d068fc-8ccc-466f-bd10-bcde02bbab90 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:59.178945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-93d068fc-8ccc-466f-bd10-bcde02bbab90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:59.181276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:59.253925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 138/281] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:22:59.636093 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c26252-1935-4959-942c-597001dedba5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:59.718061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c4de7-36be-4eb3-a46c-40f21cb5148d None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:59.718579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c4de7-36be-4eb3-a46c-40f21cb5148d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:59.718579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c4de7-36be-4eb3-a46c-40f21cb5148d None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:59.721361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 139/282] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:22:59.725357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c5a41-7b8a-4acb-b3db-d1adc0c5ed29 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:59.725636 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c5a41-7b8a-4acb-b3db-d1adc0c5ed29 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:59.725906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c5a41-7b8a-4acb-b3db-d1adc0c5ed29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:59.740813 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:22:59.758107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-3d0c5a41-7b8a-4acb-b3db-d1adc0c5ed29 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:22:59.758107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3d0c5a41-7b8a-4acb-b3db-d1adc0c5ed29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:22:59.782694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c26252-1935-4959-942c-597001dedba5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:22:59.846909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1a617-1d52-4252-94f8-2db27755ad06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:22:59.848019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1a617-1d52-4252-94f8-2db27755ad06 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:22:59.848515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1a617-1d52-4252-94f8-2db27755ad06 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:22:59.849424 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1a617-1d52-4252-94f8-2db27755ad06 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:22:59.867725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1a617-1d52-4252-94f8-2db27755ad06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-54b1a617-1d52-4252-94f8-2db27755ad06 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:22:59.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:22:59.874445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:22:59.875231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:22:59.875231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:22:59.875231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:22:59.875231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:22:59.875231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:22:59.875231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:22:59.875231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:22:59.875231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:22:59.875231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:22:59.875231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:22:59.875231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:22:59.875231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:22:59.877843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 144/283] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:22:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:22:59.883595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1a617-1d52-4252-94f8-2db27755ad06 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:22:59.892081 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 140/284] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:23:00.020585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1a617-1d52-4252-94f8-2db27755ad06 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:00.072691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016545cf-5fb7-41a8-9c5d-2e8e92036a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:00.073754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016545cf-5fb7-41a8-9c5d-2e8e92036a88 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:00.074228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016545cf-5fb7-41a8-9c5d-2e8e92036a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:00.074685 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016545cf-5fb7-41a8-9c5d-2e8e92036a88 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:00.075578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016545cf-5fb7-41a8-9c5d-2e8e92036a88 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:00.076104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016545cf-5fb7-41a8-9c5d-2e8e92036a88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:00.077706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016545cf-5fb7-41a8-9c5d-2e8e92036a88 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:00.087351 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 145/285] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:22:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:00.093380 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c5a41-7b8a-4acb-b3db-d1adc0c5ed29 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:00.193259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:00.210063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016545cf-5fb7-41a8-9c5d-2e8e92036a88 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:00.221407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:00.266569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c5a41-7b8a-4acb-b3db-d1adc0c5ed29 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:00.335203 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb3660-8779-4218-aae4-c469e386cea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:00.336281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb3660-8779-4218-aae4-c469e386cea4 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:00.336771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb3660-8779-4218-aae4-c469e386cea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:00.337505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb3660-8779-4218-aae4-c469e386cea4 None admin] PATH_INFO: `/v3/users/c78467d9763446b58050b82e57347140` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:00.340712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb3660-8779-4218-aae4-c469e386cea4 None admin] RBAC: Authorizing `identity:get_user(user_id=c78467d9763446b58050b82e57347140)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:00.342626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb3660-8779-4218-aae4-c469e386cea4 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:00.346786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 141/286] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:23:00 2022] GET /identity/v3/users/c78467d9763446b58050b82e57347140 => generated 297 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:00.350455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb3660-8779-4218-aae4-c469e386cea4 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:00.388602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016545cf-5fb7-41a8-9c5d-2e8e92036a88 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:00.457725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3962b-c27e-4f31-b263-77f942d99986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:00.458287 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3962b-c27e-4f31-b263-77f942d99986 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:00.458564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3962b-c27e-4f31-b263-77f942d99986 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:00.458833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3962b-c27e-4f31-b263-77f942d99986 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:00.479943 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3962b-c27e-4f31-b263-77f942d99986 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e8e3962b-c27e-4f31-b263-77f942d99986 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:00.485367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:00.486220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:00.487004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:00.487004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:00.487004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:00.487004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:00.487004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:00.487004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:00.487004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:00.487004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:00.487004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:00.487004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:00.487004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:00.487004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:00.487004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 146/287] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:23:00.494376 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3962b-c27e-4f31-b263-77f942d99986 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:00.530468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb3660-8779-4218-aae4-c469e386cea4 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:00.599065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb3989-8a63-42c4-9b4d-4e5cca9c628c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:00.599724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb3989-8a63-42c4-9b4d-4e5cca9c628c None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:00.600008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb3989-8a63-42c4-9b4d-4e5cca9c628c None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:00.600284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb3989-8a63-42c4-9b4d-4e5cca9c628c None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:00.602375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb3989-8a63-42c4-9b4d-4e5cca9c628c None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:00.604187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb3989-8a63-42c4-9b4d-4e5cca9c628c None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:00.609709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 142/288] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:00 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:00.611485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb3989-8a63-42c4-9b4d-4e5cca9c628c None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:00.619774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:00.643917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:00.661779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3962b-c27e-4f31-b263-77f942d99986 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:00.700555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa58f3-3128-41c1-a177-9fa98fe035fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:00.701777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa58f3-3128-41c1-a177-9fa98fe035fe None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:00.702065 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa58f3-3128-41c1-a177-9fa98fe035fe None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:00.702345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa58f3-3128-41c1-a177-9fa98fe035fe None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:00.703021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa58f3-3128-41c1-a177-9fa98fe035fe None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:00.703331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa58f3-3128-41c1-a177-9fa98fe035fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:00.707983 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa58f3-3128-41c1-a177-9fa98fe035fe None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:00.715066 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 147/289] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:23:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:00.720678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa58f3-3128-41c1-a177-9fa98fe035fe None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:00.807870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb3989-8a63-42c4-9b4d-4e5cca9c628c None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:00.843476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a180cac-3468-4512-8f6d-e50757a98bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:00.844437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a180cac-3468-4512-8f6d-e50757a98bd5 None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:00.844854 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a180cac-3468-4512-8f6d-e50757a98bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:00.845265 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a180cac-3468-4512-8f6d-e50757a98bd5 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6/users/c78467d9763446b58050b82e57347140/roles/16095b51965d46fe87bf7fc60e42bdb0` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:00.847405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a180cac-3468-4512-8f6d-e50757a98bd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ab355ce83f94db6a7c315ba27ae52a6, user_id=c78467d9763446b58050b82e57347140, role_id=16095b51965d46fe87bf7fc60e42bdb0)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:00.849276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a180cac-3468-4512-8f6d-e50757a98bd5 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:00.867533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 143/290] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:23:00 2022] PUT /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6/users/c78467d9763446b58050b82e57347140/roles/16095b51965d46fe87bf7fc60e42bdb0 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:23:00.911010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa58f3-3128-41c1-a177-9fa98fe035fe None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:00.942695 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:00.957046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28be8c4-6481-4e48-94c9-9d454ee0e500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:00.957763 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28be8c4-6481-4e48-94c9-9d454ee0e500 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:00.958099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28be8c4-6481-4e48-94c9-9d454ee0e500 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:00.958307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28be8c4-6481-4e48-94c9-9d454ee0e500 None admin] PATH_INFO: `/v3/users/833fbd75dc844396a7ccbe19696f6357` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:00.961251 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28be8c4-6481-4e48-94c9-9d454ee0e500 None admin] RBAC: Authorizing `identity:get_user(user_id=833fbd75dc844396a7ccbe19696f6357)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:00.962969 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28be8c4-6481-4e48-94c9-9d454ee0e500 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:00.964822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 148/291] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:23:00 2022] GET /identity/v3/users/833fbd75dc844396a7ccbe19696f6357 => generated 297 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:00.971064 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a180cac-3468-4512-8f6d-e50757a98bd5 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:01.057754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:01.098825 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a180cac-3468-4512-8f6d-e50757a98bd5 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:01.135920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cf061a-6620-41c6-8e33-a146f1febc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:01.136949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf061a-6620-41c6-8e33-a146f1febc65 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:01.137414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf061a-6620-41c6-8e33-a146f1febc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:01.137868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf061a-6620-41c6-8e33-a146f1febc65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:01.210143 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cf061a-6620-41c6-8e33-a146f1febc65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:01.211994 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cf061a-6620-41c6-8e33-a146f1febc65 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:01.230096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddb9e8-b17c-4162-b894-c43bf6bd8a5c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:01.230432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddb9e8-b17c-4162-b894-c43bf6bd8a5c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:01.230654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddb9e8-b17c-4162-b894-c43bf6bd8a5c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:01.232295 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 149/292] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:01.245332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677cac6-7f69-43ba-a745-e5119ea031dc None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:01.245659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677cac6-7f69-43ba-a745-e5119ea031dc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:01.246028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677cac6-7f69-43ba-a745-e5119ea031dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:01.284123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c677cac6-7f69-43ba-a745-e5119ea031dc None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:01.285328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c677cac6-7f69-43ba-a745-e5119ea031dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:01.300273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:01.358875 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 144/293] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:01.360614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41565749-ae20-45b0-8bb7-b39bdef59fa3 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:01.361095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41565749-ae20-45b0-8bb7-b39bdef59fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:01.361556 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41565749-ae20-45b0-8bb7-b39bdef59fa3 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:01.362999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 145/294] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:01.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41565749-ae20-45b0-8bb7-b39bdef59fa3 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:01.392094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 150/295] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:01.395998 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ff2c-478b-46d1-820c-cc410c049d29 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:01.395998 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ff2c-478b-46d1-820c-cc410c049d29 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:01.395998 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ff2c-478b-46d1-820c-cc410c049d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:01.415913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5ec1ff2c-478b-46d1-820c-cc410c049d29 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:01.415913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5ec1ff2c-478b-46d1-820c-cc410c049d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:01.483302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 151/296] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:01.484664 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65283d17-8393-478c-9f72-13685e84d090 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:01.484843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65283d17-8393-478c-9f72-13685e84d090 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:01.485058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65283d17-8393-478c-9f72-13685e84d090 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:01.486539 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 152/297] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:01.495465 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa1fad-1610-4341-b68d-5842386e9091 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:01.495746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa1fad-1610-4341-b68d-5842386e9091 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:01.496065 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa1fad-1610-4341-b68d-5842386e9091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:01.526400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b2fa1fad-1610-4341-b68d-5842386e9091 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:01.527593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b2fa1fad-1610-4341-b68d-5842386e9091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:01.540757 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41565749-ae20-45b0-8bb7-b39bdef59fa3 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:01.611541 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10774a6-8d8b-49e4-89c6-5be1a8d94ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:01.612222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10774a6-8d8b-49e4-89c6-5be1a8d94ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:01.612545 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10774a6-8d8b-49e4-89c6-5be1a8d94ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:01.612811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10774a6-8d8b-49e4-89c6-5be1a8d94ea9 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:01.616669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10774a6-8d8b-49e4-89c6-5be1a8d94ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:01.618292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10774a6-8d8b-49e4-89c6-5be1a8d94ea9 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:01.620485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 146/298] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:01 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:01.626260 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10774a6-8d8b-49e4-89c6-5be1a8d94ea9 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:01.626552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:01.659551 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 153/299] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:01.682013 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:01.733108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa1fad-1610-4341-b68d-5842386e9091 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:01.772398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10774a6-8d8b-49e4-89c6-5be1a8d94ea9 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:01.812045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc98116-3c66-4134-bf4a-0b1e0abd6adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:01.812674 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc98116-3c66-4134-bf4a-0b1e0abd6adc None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:01.812993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc98116-3c66-4134-bf4a-0b1e0abd6adc None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:01.813340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc98116-3c66-4134-bf4a-0b1e0abd6adc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:01.814059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc98116-3c66-4134-bf4a-0b1e0abd6adc None admin] RBAC: Adding query filter params (role.id=3100f6054f7d4d7b984fe19a4d8ba291, scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6, user.id=833fbd75dc844396a7ccbe19696f6357) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:01.814360 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc98116-3c66-4134-bf4a-0b1e0abd6adc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:01.815803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc98116-3c66-4134-bf4a-0b1e0abd6adc None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:01.822414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 147/300] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:01 2022] GET /identity/v3/role_assignments?user.id=833fbd75dc844396a7ccbe19696f6357&scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6&role.id=3100f6054f7d4d7b984fe19a4d8ba291 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:01.823367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc98116-3c66-4134-bf4a-0b1e0abd6adc None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:01.872840 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa1fad-1610-4341-b68d-5842386e9091 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:01.921224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8a97c-8973-4b9c-9a8c-257c71c9cc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:01.922304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8a97c-8973-4b9c-9a8c-257c71c9cc5a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:01.923483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8a97c-8973-4b9c-9a8c-257c71c9cc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:01.924480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8a97c-8973-4b9c-9a8c-257c71c9cc5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:01.931003 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:01.950638 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc98116-3c66-4134-bf4a-0b1e0abd6adc None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:01.953524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8a97c-8973-4b9c-9a8c-257c71c9cc5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-06b8a97c-8973-4b9c-9a8c-257c71c9cc5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:01.963824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:01.964835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:01.965602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:01.965602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:01.965602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:01.965602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:01.965602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:01.965602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:01.965602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:01.965602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:01.965602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:01.965602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:01.965602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:01.965602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:01.970781 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 154/301] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:01.976779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8a97c-8973-4b9c-9a8c-257c71c9cc5a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:02.027928 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca2f88-1404-4eec-ba06-6ae571b80642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:02.028927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca2f88-1404-4eec-ba06-6ae571b80642 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:02.029275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca2f88-1404-4eec-ba06-6ae571b80642 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:02.029607 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca2f88-1404-4eec-ba06-6ae571b80642 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:02.041481 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca2f88-1404-4eec-ba06-6ae571b80642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-f2ca2f88-1404-4eec-ba06-6ae571b80642 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:02.047224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:02.048181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:02.049369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:02.049369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:02.049369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:02.049369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:02.049369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:02.049369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:02.049369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:02.049369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:02.049369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:02.049369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:02.049369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:23:02.049369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:02.049369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 148/302] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:02.053100 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca2f88-1404-4eec-ba06-6ae571b80642 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:02.054485 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:02.132902 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8a97c-8973-4b9c-9a8c-257c71c9cc5a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:02.174414 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb469240-fa66-4d9d-9860-3ce05728797e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:02.175034 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb469240-fa66-4d9d-9860-3ce05728797e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:02.175306 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb469240-fa66-4d9d-9860-3ce05728797e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:02.175575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb469240-fa66-4d9d-9860-3ce05728797e None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:02.176327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb469240-fa66-4d9d-9860-3ce05728797e None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:02.176648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb469240-fa66-4d9d-9860-3ce05728797e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:02.178014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb469240-fa66-4d9d-9860-3ce05728797e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:02.183037 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca2f88-1404-4eec-ba06-6ae571b80642 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:02.186747 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 155/303] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:23:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:02.187658 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb469240-fa66-4d9d-9860-3ce05728797e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:02.221980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365ae04-a36c-428a-8acb-f581a7ff4726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:02.222921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365ae04-a36c-428a-8acb-f581a7ff4726 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:02.223608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365ae04-a36c-428a-8acb-f581a7ff4726 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:02.224117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365ae04-a36c-428a-8acb-f581a7ff4726 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:02.226257 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365ae04-a36c-428a-8acb-f581a7ff4726 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:02.227977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365ae04-a36c-428a-8acb-f581a7ff4726 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:02.230250 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 149/304] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:02.231181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365ae04-a36c-428a-8acb-f581a7ff4726 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:02.261406 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:02.296113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb469240-fa66-4d9d-9860-3ce05728797e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:02.306103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:02.332488 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59428f13-2a61-4976-a780-f50f0adee672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:02.333363 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59428f13-2a61-4976-a780-f50f0adee672 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:02.333648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59428f13-2a61-4976-a780-f50f0adee672 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:02.333900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59428f13-2a61-4976-a780-f50f0adee672 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:02.334561 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59428f13-2a61-4976-a780-f50f0adee672 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:02.334855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59428f13-2a61-4976-a780-f50f0adee672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:02.336248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59428f13-2a61-4976-a780-f50f0adee672 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:02.343616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365ae04-a36c-428a-8acb-f581a7ff4726 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:02.349178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 156/305] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:23:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:02.349178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59428f13-2a61-4976-a780-f50f0adee672 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:02.384896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd615335-431d-4880-bf52-9e882250d195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:02.385533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd615335-431d-4880-bf52-9e882250d195 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:02.385808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd615335-431d-4880-bf52-9e882250d195 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:02.386070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd615335-431d-4880-bf52-9e882250d195 None admin] PATH_INFO: `/v3/users/87a3aa28af424890829a6cfd63905975` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:02.387627 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd615335-431d-4880-bf52-9e882250d195 None admin] RBAC: Authorizing `identity:get_user(user_id=87a3aa28af424890829a6cfd63905975)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:02.389458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd615335-431d-4880-bf52-9e882250d195 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:02.391527 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 150/306] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:23:02 2022] GET /identity/v3/users/87a3aa28af424890829a6cfd63905975 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:02.392122 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd615335-431d-4880-bf52-9e882250d195 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:02.454212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59428f13-2a61-4976-a780-f50f0adee672 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:02.489335 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:02.491553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fcee3-ff93-442e-a6ea-292d35240b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:02.492209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fcee3-ff93-442e-a6ea-292d35240b48 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:02.492446 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fcee3-ff93-442e-a6ea-292d35240b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:02.492734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fcee3-ff93-442e-a6ea-292d35240b48 None admin] PATH_INFO: `/v3/groups` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:02.493427 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7fcee3-ff93-442e-a6ea-292d35240b48 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:02.495064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7fcee3-ff93-442e-a6ea-292d35240b48 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:02.500037 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd615335-431d-4880-bf52-9e882250d195 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:02.513645 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 157/307] 10.222.0.62 () {60 vars in 1134 bytes} [Thu Sep 1 16:23:02 2022] POST /identity/v3/groups => generated 229 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:23:02.514287 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fcee3-ff93-442e-a6ea-292d35240b48 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:02.549213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984fc46-d8b0-4798-96d7-e31c8f988225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:02.550664 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:02.550802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984fc46-d8b0-4798-96d7-e31c8f988225 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:02.551279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984fc46-d8b0-4798-96d7-e31c8f988225 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:02.552291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984fc46-d8b0-4798-96d7-e31c8f988225 None admin] PATH_INFO: `/v3/users/90b7de9cd6e64ec7a293d478020a0af3` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:02.575656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984fc46-d8b0-4798-96d7-e31c8f988225 None admin] RBAC: Authorizing `identity:get_user(user_id=90b7de9cd6e64ec7a293d478020a0af3)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:02.577290 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984fc46-d8b0-4798-96d7-e31c8f988225 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:02.580032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 151/308] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:23:02 2022] GET /identity/v3/users/90b7de9cd6e64ec7a293d478020a0af3 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:02.587586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984fc46-d8b0-4798-96d7-e31c8f988225 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:02.638613 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fcee3-ff93-442e-a6ea-292d35240b48 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:02.680386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa902900-7440-40a2-867e-ce035df66296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:02.681320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa902900-7440-40a2-867e-ce035df66296 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:02.681703 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa902900-7440-40a2-867e-ce035df66296 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:02.681975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa902900-7440-40a2-867e-ce035df66296 None admin] PATH_INFO: `/v3/system/users/87a3aa28af424890829a6cfd63905975/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:02.683621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa902900-7440-40a2-867e-ce035df66296 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=87a3aa28af424890829a6cfd63905975, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:02.685095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa902900-7440-40a2-867e-ce035df66296 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:02.700806 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 158/309] 10.222.0.62 () {58 vars in 1334 bytes} [Thu Sep 1 16:23:02 2022] PUT /identity/v3/system/users/87a3aa28af424890829a6cfd63905975/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:23:02.736233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984fc46-d8b0-4798-96d7-e31c8f988225 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:02.789523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b88f8-010c-4611-8cac-f18deb0c11f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:02.790642 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b88f8-010c-4611-8cac-f18deb0c11f0 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:02.791160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b88f8-010c-4611-8cac-f18deb0c11f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:02.791563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b88f8-010c-4611-8cac-f18deb0c11f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:02.792505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b88f8-010c-4611-8cac-f18deb0c11f0 None admin] RBAC: Adding query filter params (role.id=16095b51965d46fe87bf7fc60e42bdb0, user.id=90b7de9cd6e64ec7a293d478020a0af3, scope.system=all) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:02.792968 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b88f8-010c-4611-8cac-f18deb0c11f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:02.794609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b88f8-010c-4611-8cac-f18deb0c11f0 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:02.801463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 152/310] 10.222.0.62 () {56 vars in 1308 bytes} [Thu Sep 1 16:23:02 2022] GET /identity/v3/role_assignments?user.id=90b7de9cd6e64ec7a293d478020a0af3&scope.system=all&role.id=16095b51965d46fe87bf7fc60e42bdb0 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:03.119229 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6371f-ed4f-4c28-b240-624c501e34ac None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:03.119644 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6371f-ed4f-4c28-b240-624c501e34ac None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:03.119764 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6371f-ed4f-4c28-b240-624c501e34ac None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:03.121398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 159/311] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:03.126897 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7c0c4-e900-4524-9c4d-ee5a15747cb6 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:03.127346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7c0c4-e900-4524-9c4d-ee5a15747cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:03.127419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7c0c4-e900-4524-9c4d-ee5a15747cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:03.146050 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-72a7c0c4-e900-4524-9c4d-ee5a15747cb6 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:03.147252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-72a7c0c4-e900-4524-9c4d-ee5a15747cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:03.157194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:03.216555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 153/312] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:03.482746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6371f-ed4f-4c28-b240-624c501e34ac None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:03.575904 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:03.584406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6371f-ed4f-4c28-b240-624c501e34ac None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:03.620249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51232944-6f06-4807-8417-f11519dea2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:03.620896 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51232944-6f06-4807-8417-f11519dea2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:03.621169 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51232944-6f06-4807-8417-f11519dea2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:03.621437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51232944-6f06-4807-8417-f11519dea2d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:03.632104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51232944-6f06-4807-8417-f11519dea2d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-51232944-6f06-4807-8417-f11519dea2d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:03.637268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:03.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:03.638951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:03.638951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:03.638951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:03.638951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:03.638951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:03.638951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:03.638951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:03.638951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:03.638951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:03.638951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:03.638951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:23:03.638951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:03.638951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 160/313] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:03.644319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7c0c4-e900-4524-9c4d-ee5a15747cb6 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:03.768793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7c0c4-e900-4524-9c4d-ee5a15747cb6 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:03.808370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b038d-6d59-49cb-aae3-fcfb2791269f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:03.809012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b038d-6d59-49cb-aae3-fcfb2791269f None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:03.809286 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b038d-6d59-49cb-aae3-fcfb2791269f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:03.809551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b038d-6d59-49cb-aae3-fcfb2791269f None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:03.810229 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b038d-6d59-49cb-aae3-fcfb2791269f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:03.810526 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b038d-6d59-49cb-aae3-fcfb2791269f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:03.813452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b038d-6d59-49cb-aae3-fcfb2791269f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:03.821205 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 154/314] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:23:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:03.827558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51232944-6f06-4807-8417-f11519dea2d4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:03.933883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51232944-6f06-4807-8417-f11519dea2d4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:03.969352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d2a02-bea2-4fc0-8889-5befe092c9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:03.970096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d2a02-bea2-4fc0-8889-5befe092c9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:03.970436 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d2a02-bea2-4fc0-8889-5befe092c9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:03.970775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d2a02-bea2-4fc0-8889-5befe092c9c1 None admin] PATH_INFO: `/v3/users/c78467d9763446b58050b82e57347140` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:03.972078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d2a02-bea2-4fc0-8889-5befe092c9c1 None admin] RBAC: Authorizing `identity:get_user(user_id=c78467d9763446b58050b82e57347140)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:03.973658 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d2a02-bea2-4fc0-8889-5befe092c9c1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:03.975497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 161/315] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:23:03 2022] GET /identity/v3/users/c78467d9763446b58050b82e57347140 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:03.981788 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b038d-6d59-49cb-aae3-fcfb2791269f None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:03.994413 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:04.094684 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b038d-6d59-49cb-aae3-fcfb2791269f None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:04.128454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338b47c-9a14-4f64-a178-6d712e64792e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:04.129073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338b47c-9a14-4f64-a178-6d712e64792e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:04.129323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338b47c-9a14-4f64-a178-6d712e64792e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:04.129588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338b47c-9a14-4f64-a178-6d712e64792e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:04.197375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338b47c-9a14-4f64-a178-6d712e64792e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:04.198901 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338b47c-9a14-4f64-a178-6d712e64792e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:04.207344 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:04.300892 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 155/316] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:04.306888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d2a02-bea2-4fc0-8889-5befe092c9c1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:04.412451 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:04.418145 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d2a02-bea2-4fc0-8889-5befe092c9c1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:04.454073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e85f32-f7d8-4e27-94be-15101eed9b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:04.454819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e85f32-f7d8-4e27-94be-15101eed9b83 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:04.455165 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e85f32-f7d8-4e27-94be-15101eed9b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:04.455505 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e85f32-f7d8-4e27-94be-15101eed9b83 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:04.457086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e85f32-f7d8-4e27-94be-15101eed9b83 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:04.459164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e85f32-f7d8-4e27-94be-15101eed9b83 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:04.465877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 162/317] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:04 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:04.469042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338b47c-9a14-4f64-a178-6d712e64792e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:04.570308 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338b47c-9a14-4f64-a178-6d712e64792e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:04.574697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fd4da-db73-4a87-8d9e-6ad4c5affcdf None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:04.574980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fd4da-db73-4a87-8d9e-6ad4c5affcdf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:04.575254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fd4da-db73-4a87-8d9e-6ad4c5affcdf None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:04.577902 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 163/318] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:23:04.578173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:04.583036 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff412f-628b-4af3-955f-94ce00aebdf0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:04.583316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff412f-628b-4af3-955f-94ce00aebdf0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:04.583593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff412f-628b-4af3-955f-94ce00aebdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:04.606147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684c8c2-e4b1-4130-818a-968fb56a51a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:04.606494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b0ff412f-628b-4af3-955f-94ce00aebdf0 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:04.606751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684c8c2-e4b1-4130-818a-968fb56a51a1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:04.607007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684c8c2-e4b1-4130-818a-968fb56a51a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:04.607272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684c8c2-e4b1-4130-818a-968fb56a51a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:04.607612 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b0ff412f-628b-4af3-955f-94ce00aebdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:04.608079 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684c8c2-e4b1-4130-818a-968fb56a51a1 None admin] RBAC: Adding query filter params (role.id=16095b51965d46fe87bf7fc60e42bdb0, scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6, user.id=c78467d9763446b58050b82e57347140) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:04.608379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684c8c2-e4b1-4130-818a-968fb56a51a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:04.609825 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684c8c2-e4b1-4130-818a-968fb56a51a1 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:04.616160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 156/319] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:04 2022] GET /identity/v3/role_assignments?user.id=c78467d9763446b58050b82e57347140&scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6&role.id=16095b51965d46fe87bf7fc60e42bdb0 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:04.671494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ee1c7-db9e-4ed3-8f0e-204f443f9e63 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:04.671780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ee1c7-db9e-4ed3-8f0e-204f443f9e63 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:04.671988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ee1c7-db9e-4ed3-8f0e-204f443f9e63 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:04.673649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 157/320] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:23:04.679302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b26093-f037-442e-b6b5-92210b929aec None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:04.679941 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b26093-f037-442e-b6b5-92210b929aec None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:04.680416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b26093-f037-442e-b6b5-92210b929aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:04.698587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 164/321] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:23:04.708358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e9b26093-f037-442e-b6b5-92210b929aec None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:04.709931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e9b26093-f037-442e-b6b5-92210b929aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:04.793012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 158/322] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:23:04.862972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf3b8a-ec4f-4e22-b7bd-22c5b872cbef None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:04.863496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf3b8a-ec4f-4e22-b7bd-22c5b872cbef None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:04.863496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf3b8a-ec4f-4e22-b7bd-22c5b872cbef None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:04.865150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 165/323] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:04.870490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bd2ed-417c-4c8b-8a17-9981209a6aa5 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:04.870772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bd2ed-417c-4c8b-8a17-9981209a6aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:04.871046 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bd2ed-417c-4c8b-8a17-9981209a6aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:04.890724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e57bd2ed-417c-4c8b-8a17-9981209a6aa5 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:04.892232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e57bd2ed-417c-4c8b-8a17-9981209a6aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:04.964012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 159/324] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:04.965297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf3b8a-ec4f-4e22-b7bd-22c5b872cbef None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:04.991295 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:05.066621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bd2ed-417c-4c8b-8a17-9981209a6aa5 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:05.075196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf3b8a-ec4f-4e22-b7bd-22c5b872cbef None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:05.101602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:05.112202 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48070b7d-2a68-42ee-9cec-823cc11e6052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:05.112838 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48070b7d-2a68-42ee-9cec-823cc11e6052 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:05.113109 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48070b7d-2a68-42ee-9cec-823cc11e6052 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:05.113370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48070b7d-2a68-42ee-9cec-823cc11e6052 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:05.114452 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48070b7d-2a68-42ee-9cec-823cc11e6052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:05.115905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48070b7d-2a68-42ee-9cec-823cc11e6052 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:05.117652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 166/325] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:05.122931 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48070b7d-2a68-42ee-9cec-823cc11e6052 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:05.178572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bd2ed-417c-4c8b-8a17-9981209a6aa5 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:05.217482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:05.222145 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7fb2e-1855-4104-94dd-407e6ba36938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:05.222903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7fb2e-1855-4104-94dd-407e6ba36938 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:05.223248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7fb2e-1855-4104-94dd-407e6ba36938 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:05.223675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7fb2e-1855-4104-94dd-407e6ba36938 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:05.231077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48070b7d-2a68-42ee-9cec-823cc11e6052 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:05.234295 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7fb2e-1855-4104-94dd-407e6ba36938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-9ce7fb2e-1855-4104-94dd-407e6ba36938 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:05.239266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:05.240358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:05.241240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:05.241240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:05.241240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:05.241240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:05.241240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:05.241240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:05.241240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:05.241240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:05.241240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:05.241240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:05.241240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:05.241240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:05.241240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 160/326] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:05.241240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7fb2e-1855-4104-94dd-407e6ba36938 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:05.268630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2ec93-182e-42c5-b5d4-7d60b2a0f67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:05.269497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2ec93-182e-42c5-b5d4-7d60b2a0f67e None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:05.269770 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2ec93-182e-42c5-b5d4-7d60b2a0f67e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:05.270040 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2ec93-182e-42c5-b5d4-7d60b2a0f67e None admin] PATH_INFO: `/v3/groups` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:05.270655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2ec93-182e-42c5-b5d4-7d60b2a0f67e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:05.272193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2ec93-182e-42c5-b5d4-7d60b2a0f67e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:05.288124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 167/327] 10.222.0.62 () {60 vars in 1134 bytes} [Thu Sep 1 16:23:05 2022] POST /identity/v3/groups => generated 226 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:23:05.288941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2ec93-182e-42c5-b5d4-7d60b2a0f67e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:05.356584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:05.368423 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7fb2e-1855-4104-94dd-407e6ba36938 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:05.405588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2ec93-182e-42c5-b5d4-7d60b2a0f67e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:05.407867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6f1bd-2bc7-4c9d-8dca-bac652a02e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:05.408552 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6f1bd-2bc7-4c9d-8dca-bac652a02e7a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:05.408823 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6f1bd-2bc7-4c9d-8dca-bac652a02e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:05.409098 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6f1bd-2bc7-4c9d-8dca-bac652a02e7a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:05.420802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6f1bd-2bc7-4c9d-8dca-bac652a02e7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-a3d6f1bd-2bc7-4c9d-8dca-bac652a02e7a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:05.425769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:05.426687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:05.427516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:05.427516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:05.427516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:05.427516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:05.427516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:05.427516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:05.427516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:05.427516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:05.427516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:05.427516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:05.427516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:23:05.427516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:05.427516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 161/328] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:05.433333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6f1bd-2bc7-4c9d-8dca-bac652a02e7a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:05.458686 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124eb5a-d511-4c2c-93cb-31ba48e9af7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:05.459568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124eb5a-d511-4c2c-93cb-31ba48e9af7d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:05.459568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124eb5a-d511-4c2c-93cb-31ba48e9af7d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:05.459846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124eb5a-d511-4c2c-93cb-31ba48e9af7d None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:05.461475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124eb5a-d511-4c2c-93cb-31ba48e9af7d None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:05.461475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124eb5a-d511-4c2c-93cb-31ba48e9af7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:05.462322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124eb5a-d511-4c2c-93cb-31ba48e9af7d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:05.472242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 168/329] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:23:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:05.477632 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124eb5a-d511-4c2c-93cb-31ba48e9af7d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:05.494112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:05.571597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6f1bd-2bc7-4c9d-8dca-bac652a02e7a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:05.602055 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124eb5a-d511-4c2c-93cb-31ba48e9af7d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:05.619239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c820b2-ecbc-45e2-8155-c2a20a6d46c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:05.620104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c820b2-ecbc-45e2-8155-c2a20a6d46c4 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:05.620482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c820b2-ecbc-45e2-8155-c2a20a6d46c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:05.620852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c820b2-ecbc-45e2-8155-c2a20a6d46c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:05.621625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c820b2-ecbc-45e2-8155-c2a20a6d46c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:05.622037 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c820b2-ecbc-45e2-8155-c2a20a6d46c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:05.623519 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c820b2-ecbc-45e2-8155-c2a20a6d46c4 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:05.633888 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 162/330] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:23:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:05.640035 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c820b2-ecbc-45e2-8155-c2a20a6d46c4 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:05.642851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75df5f-0799-4050-a83a-3502e6c01ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:05.643776 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75df5f-0799-4050-a83a-3502e6c01ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:05.644077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75df5f-0799-4050-a83a-3502e6c01ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:05.646432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75df5f-0799-4050-a83a-3502e6c01ce8 None admin] PATH_INFO: `/v3/users/87a3aa28af424890829a6cfd63905975` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:05.646432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f75df5f-0799-4050-a83a-3502e6c01ce8 None admin] RBAC: Authorizing `identity:get_user(user_id=87a3aa28af424890829a6cfd63905975)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:05.647863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f75df5f-0799-4050-a83a-3502e6c01ce8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:05.649751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 169/331] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:23:05 2022] GET /identity/v3/users/87a3aa28af424890829a6cfd63905975 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:05.655918 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75df5f-0799-4050-a83a-3502e6c01ce8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:05.666984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:05.746083 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:05.769784 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c820b2-ecbc-45e2-8155-c2a20a6d46c4 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:05.770446 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75df5f-0799-4050-a83a-3502e6c01ce8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:05.806231 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf24673-44a0-4615-b305-9aa60e9d176e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:05.806816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf24673-44a0-4615-b305-9aa60e9d176e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:05.807090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf24673-44a0-4615-b305-9aa60e9d176e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:05.807357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf24673-44a0-4615-b305-9aa60e9d176e None admin] PATH_INFO: `/v3/users/90b7de9cd6e64ec7a293d478020a0af3` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:05.807932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049ee85-b192-46ac-beca-2b1b2464643b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:05.808503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf24673-44a0-4615-b305-9aa60e9d176e None admin] RBAC: Authorizing `identity:get_user(user_id=90b7de9cd6e64ec7a293d478020a0af3)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:05.808762 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049ee85-b192-46ac-beca-2b1b2464643b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:05.809137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049ee85-b192-46ac-beca-2b1b2464643b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:05.809504 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049ee85-b192-46ac-beca-2b1b2464643b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:05.809954 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf24673-44a0-4615-b305-9aa60e9d176e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:05.810422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049ee85-b192-46ac-beca-2b1b2464643b None admin] RBAC: Adding query filter params (role.id=3100f6054f7d4d7b984fe19a4d8ba291, user.id=87a3aa28af424890829a6cfd63905975, scope.system=all) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:05.810827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049ee85-b192-46ac-beca-2b1b2464643b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:05.811958 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 163/332] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:23:05 2022] GET /identity/v3/users/90b7de9cd6e64ec7a293d478020a0af3 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:05.812444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049ee85-b192-46ac-beca-2b1b2464643b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:05.820615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf24673-44a0-4615-b305-9aa60e9d176e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:05.833476 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 170/333] 10.222.0.62 () {56 vars in 1308 bytes} [Thu Sep 1 16:23:05 2022] GET /identity/v3/role_assignments?user.id=87a3aa28af424890829a6cfd63905975&scope.system=all&role.id=3100f6054f7d4d7b984fe19a4d8ba291 => generated 510 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:05.925500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:05.947631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf24673-44a0-4615-b305-9aa60e9d176e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:05.988016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e43a0-5e8a-4e05-a131-2468ccb761fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:05.988698 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e43a0-5e8a-4e05-a131-2468ccb761fd None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:05.988976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e43a0-5e8a-4e05-a131-2468ccb761fd None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:05.989250 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e43a0-5e8a-4e05-a131-2468ccb761fd None admin] PATH_INFO: `/v3/system/users/90b7de9cd6e64ec7a293d478020a0af3/roles/16095b51965d46fe87bf7fc60e42bdb0` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:05.991352 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e43a0-5e8a-4e05-a131-2468ccb761fd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=90b7de9cd6e64ec7a293d478020a0af3, role_id=16095b51965d46fe87bf7fc60e42bdb0)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:05.992124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e43a0-5e8a-4e05-a131-2468ccb761fd None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:06.004189 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 164/334] 10.222.0.62 () {58 vars in 1334 bytes} [Thu Sep 1 16:23:05 2022] PUT /identity/v3/system/users/90b7de9cd6e64ec7a293d478020a0af3/roles/16095b51965d46fe87bf7fc60e42bdb0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:23:07.424915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605f347-f378-4062-bef5-1ef2e2093146 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:07.425467 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605f347-f378-4062-bef5-1ef2e2093146 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:07.425467 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605f347-f378-4062-bef5-1ef2e2093146 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:07.427042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 171/335] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:07.432448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acc094-18ac-4cee-8ec4-57d543fcb5f8 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:07.432869 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acc094-18ac-4cee-8ec4-57d543fcb5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:07.433110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acc094-18ac-4cee-8ec4-57d543fcb5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:07.453647 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-a1acc094-18ac-4cee-8ec4-57d543fcb5f8 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:07.454838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a1acc094-18ac-4cee-8ec4-57d543fcb5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:07.530851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 165/336] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:07.857037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605f347-f378-4062-bef5-1ef2e2093146 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:07.971027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605f347-f378-4062-bef5-1ef2e2093146 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:08.007853 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b8dfa-76cf-4ede-9c76-510997925b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:08.008524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b8dfa-76cf-4ede-9c76-510997925b9d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:08.008801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b8dfa-76cf-4ede-9c76-510997925b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:08.009314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b8dfa-76cf-4ede-9c76-510997925b9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:08.014074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:08.024838 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b8dfa-76cf-4ede-9c76-510997925b9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-f84b8dfa-76cf-4ede-9c76-510997925b9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:08.030139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:08.031193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:08.032196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:08.032196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:08.032196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:08.032196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:08.032196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:08.032196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:08.032196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:08.032196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:08.032196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:08.032196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:08.032196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:08.032196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:08.034770 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 172/337] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:08.040638 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acc094-18ac-4cee-8ec4-57d543fcb5f8 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:08.125287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:08.142025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe503-5d6f-423a-b59b-77af48288e57 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:08.142334 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe503-5d6f-423a-b59b-77af48288e57 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:08.142570 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe503-5d6f-423a-b59b-77af48288e57 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:08.144523 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 173/338] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:23:08.149692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637991f-737d-4b86-abf2-d3f2163cfdbb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:08.149984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637991f-737d-4b86-abf2-d3f2163cfdbb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:08.150257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637991f-737d-4b86-abf2-d3f2163cfdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:08.172924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acc094-18ac-4cee-8ec4-57d543fcb5f8 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:08.189234 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5637991f-737d-4b86-abf2-d3f2163cfdbb None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:08.190803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5637991f-737d-4b86-abf2-d3f2163cfdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:08.246178 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcdc31d-7f97-4d1e-b3cc-b670f39889c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:08.246810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdc31d-7f97-4d1e-b3cc-b670f39889c1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:08.247082 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdc31d-7f97-4d1e-b3cc-b670f39889c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:08.247346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcdc31d-7f97-4d1e-b3cc-b670f39889c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:08.248822 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcdc31d-7f97-4d1e-b3cc-b670f39889c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:08.249129 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcdc31d-7f97-4d1e-b3cc-b670f39889c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:08.250533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcdc31d-7f97-4d1e-b3cc-b670f39889c1 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:08.276347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 166/339] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:23:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:08.282616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcdc31d-7f97-4d1e-b3cc-b670f39889c1 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:08.340268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 174/340] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:23:08.440065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcdc31d-7f97-4d1e-b3cc-b670f39889c1 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:08.452980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29143ec7-c5b6-4c4e-b6e5-c7d0b885c5dd None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:08.453498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29143ec7-c5b6-4c4e-b6e5-c7d0b885c5dd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:08.453498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29143ec7-c5b6-4c4e-b6e5-c7d0b885c5dd None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:08.459394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 175/341] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:23:08.461302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1ca7f-7d8a-4160-b533-f63516699c45 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:08.461587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1ca7f-7d8a-4160-b533-f63516699c45 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:08.461859 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1ca7f-7d8a-4160-b533-f63516699c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:08.493109 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9fb1ca7f-7d8a-4160-b533-f63516699c45 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:08.494593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9fb1ca7f-7d8a-4160-b533-f63516699c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:08.502013 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c4b72-4eac-4476-9822-64718fe387bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:08.502920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c4b72-4eac-4476-9822-64718fe387bb None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:08.503416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c4b72-4eac-4476-9822-64718fe387bb None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:08.503869 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c4b72-4eac-4476-9822-64718fe387bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:08.514773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:08.602670 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 176/342] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:08.643371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c4b72-4eac-4476-9822-64718fe387bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:08.644876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c4b72-4eac-4476-9822-64718fe387bb None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:08.771548 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 167/343] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:23:08.777763 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1ca7f-7d8a-4160-b533-f63516699c45 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:08.849738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c4b72-4eac-4476-9822-64718fe387bb None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:08.850591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:08.892590 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:08.935442 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1ca7f-7d8a-4160-b533-f63516699c45 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:09.002958 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c4b72-4eac-4476-9822-64718fe387bb None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:09.006604 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33667e-b0eb-41d4-a8e0-0fc763fe5f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:09.007509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33667e-b0eb-41d4-a8e0-0fc763fe5f10 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:09.008090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33667e-b0eb-41d4-a8e0-0fc763fe5f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:09.008581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33667e-b0eb-41d4-a8e0-0fc763fe5f10 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:09.010362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33667e-b0eb-41d4-a8e0-0fc763fe5f10 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:09.013050 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33667e-b0eb-41d4-a8e0-0fc763fe5f10 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:09.015275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 177/344] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:08 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:09.019909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a08e8a-8676-4781-acec-612da4e6073b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:09.019909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a08e8a-8676-4781-acec-612da4e6073b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:09.019909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a08e8a-8676-4781-acec-612da4e6073b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:09.021760 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 178/345] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:09.021987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a08e8a-8676-4781-acec-612da4e6073b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:09.063728 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679f896-21ba-4d1d-9051-b6eed24ccb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:09.064532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679f896-21ba-4d1d-9051-b6eed24ccb17 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:09.064811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679f896-21ba-4d1d-9051-b6eed24ccb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:09.065075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679f896-21ba-4d1d-9051-b6eed24ccb17 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:09.080360 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679f896-21ba-4d1d-9051-b6eed24ccb17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-1679f896-21ba-4d1d-9051-b6eed24ccb17 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:09.087114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:09.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:09.089169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:09.089169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:09.089169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:09.089169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:09.089169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:09.089169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:09.089169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:09.089169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:09.089169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:09.089169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:09.089169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:23:09.089169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:09.089169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 168/346] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:23:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:09.089808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679f896-21ba-4d1d-9051-b6eed24ccb17 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:09.149616 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a08e8a-8676-4781-acec-612da4e6073b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:09.177627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:09.191123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f05d1d-4489-4a81-8576-a7055305d191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:09.192270 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f05d1d-4489-4a81-8576-a7055305d191 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:09.192856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f05d1d-4489-4a81-8576-a7055305d191 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:09.193322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f05d1d-4489-4a81-8576-a7055305d191 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:09.204344 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:09.210473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f05d1d-4489-4a81-8576-a7055305d191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:09.212238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679f896-21ba-4d1d-9051-b6eed24ccb17 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c2f05d1d-4489-4a81-8576-a7055305d191 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:09.216466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:09.217428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:09.218240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:09.218240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:09.218240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:09.218240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:09.218240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:09.218240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:09.218240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:09.218240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:09.218240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:09.218240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:09.218240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:09.218240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:09.220540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 179/347] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:09.222905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7db2f-0d12-4df0-b7b2-d69b4f200013 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:09.223401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7db2f-0d12-4df0-b7b2-d69b4f200013 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:09.223877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7db2f-0d12-4df0-b7b2-d69b4f200013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:09.245590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-17e7db2f-0d12-4df0-b7b2-d69b4f200013 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:09.246975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-17e7db2f-0d12-4df0-b7b2-d69b4f200013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:09.253426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f377b1c-88b9-4893-8ecb-1c4960305adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:09.254290 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f377b1c-88b9-4893-8ecb-1c4960305adb None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:09.254702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f377b1c-88b9-4893-8ecb-1c4960305adb None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:09.255167 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f377b1c-88b9-4893-8ecb-1c4960305adb None admin] PATH_INFO: `/v3/groups/459bcbad31f44b8fad10cdd5de14856c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:09.263643 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f377b1c-88b9-4893-8ecb-1c4960305adb None admin] RBAC: Authorizing `identity:get_group(group_id=459bcbad31f44b8fad10cdd5de14856c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:09.266119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f377b1c-88b9-4893-8ecb-1c4960305adb None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:09.268684 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 169/348] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:09 2022] GET /identity/v3/groups/459bcbad31f44b8fad10cdd5de14856c => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:09.269695 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f377b1c-88b9-4893-8ecb-1c4960305adb None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:09.329608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 180/349] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:09.329608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7db2f-0d12-4df0-b7b2-d69b4f200013 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:09.414824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f377b1c-88b9-4893-8ecb-1c4960305adb None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:09.476631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360d735-6ce5-4442-b74a-b6ea744f2f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:09.478087 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360d735-6ce5-4442-b74a-b6ea744f2f5f None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:09.478575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360d735-6ce5-4442-b74a-b6ea744f2f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:09.479016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360d735-6ce5-4442-b74a-b6ea744f2f5f None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:09.479867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360d735-6ce5-4442-b74a-b6ea744f2f5f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:09.480687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360d735-6ce5-4442-b74a-b6ea744f2f5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:09.485132 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360d735-6ce5-4442-b74a-b6ea744f2f5f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:09.498943 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 170/350] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:09.499830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360d735-6ce5-4442-b74a-b6ea744f2f5f None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:09.527942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:09.529084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7db2f-0d12-4df0-b7b2-d69b4f200013 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:09.547514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:09.571807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b84d3-3ecb-47c7-bde7-70619f0d5561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:09.572838 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b84d3-3ecb-47c7-bde7-70619f0d5561 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:09.573352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b84d3-3ecb-47c7-bde7-70619f0d5561 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:09.573846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b84d3-3ecb-47c7-bde7-70619f0d5561 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:09.574741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b84d3-3ecb-47c7-bde7-70619f0d5561 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:09.575213 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b84d3-3ecb-47c7-bde7-70619f0d5561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:09.576930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b84d3-3ecb-47c7-bde7-70619f0d5561 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:09.591025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 181/351] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:23:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:09.591025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f7ab4-fdc4-43df-b949-39bd6feb6a60 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:09.591025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f7ab4-fdc4-43df-b949-39bd6feb6a60 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:09.591025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f7ab4-fdc4-43df-b949-39bd6feb6a60 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:09.591408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 182/352] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:09.592140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f7ab4-fdc4-43df-b949-39bd6feb6a60 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:09.622060 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360d735-6ce5-4442-b74a-b6ea744f2f5f None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:09.667486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a17979-e46b-4d33-9a49-e972e7336f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:09.668078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a17979-e46b-4d33-9a49-e972e7336f6e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:09.668382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a17979-e46b-4d33-9a49-e972e7336f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:09.668657 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a17979-e46b-4d33-9a49-e972e7336f6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:09.669445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a17979-e46b-4d33-9a49-e972e7336f6e None admin] RBAC: Adding query filter params (group.id=459bcbad31f44b8fad10cdd5de14856c, role.id=3100f6054f7d4d7b984fe19a4d8ba291, scope.project.id=ce0bf520fc554803bd196782f428d532) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:09.670575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a17979-e46b-4d33-9a49-e972e7336f6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:09.671266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a17979-e46b-4d33-9a49-e972e7336f6e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:09.677178 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 171/353] 10.222.0.62 () {56 vars in 1376 bytes} [Thu Sep 1 16:23:09 2022] GET /identity/v3/role_assignments?group.id=459bcbad31f44b8fad10cdd5de14856c&scope.project.id=ce0bf520fc554803bd196782f428d532&role.id=3100f6054f7d4d7b984fe19a4d8ba291 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:09.678001 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a17979-e46b-4d33-9a49-e972e7336f6e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:09.729948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f7ab4-fdc4-43df-b949-39bd6feb6a60 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:09.787296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830e288-9ccf-47f7-bd75-7ffa1124dc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:09.788405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830e288-9ccf-47f7-bd75-7ffa1124dc0f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:09.788972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830e288-9ccf-47f7-bd75-7ffa1124dc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:09.789553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830e288-9ccf-47f7-bd75-7ffa1124dc0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:09.803268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:09.824802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a17979-e46b-4d33-9a49-e972e7336f6e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:09.851945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:09.877475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba190d1-e6bb-448c-b8be-c27da5277d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:09.878107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba190d1-e6bb-448c-b8be-c27da5277d9a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:09.878385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba190d1-e6bb-448c-b8be-c27da5277d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:09.878663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba190d1-e6bb-448c-b8be-c27da5277d9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:09.889252 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830e288-9ccf-47f7-bd75-7ffa1124dc0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:09.891200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830e288-9ccf-47f7-bd75-7ffa1124dc0f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:09.975975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba190d1-e6bb-448c-b8be-c27da5277d9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:09.977959 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba190d1-e6bb-448c-b8be-c27da5277d9a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:10.026800 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 183/354] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:10.028209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421965c-e393-4f1a-a2fe-5f332cfc3427 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:10.028497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421965c-e393-4f1a-a2fe-5f332cfc3427 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:10.028771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421965c-e393-4f1a-a2fe-5f332cfc3427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:10.052118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4421965c-e393-4f1a-a2fe-5f332cfc3427 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:10.053824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4421965c-e393-4f1a-a2fe-5f332cfc3427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:10.079552 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:10.104930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 172/355] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:10.105895 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba190d1-e6bb-448c-b8be-c27da5277d9a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:10.146602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 184/356] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:10.147475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421965c-e393-4f1a-a2fe-5f332cfc3427 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:10.158106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:10.237859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba190d1-e6bb-448c-b8be-c27da5277d9a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:10.289128 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407c527-9101-4d72-a76b-d8fc718fbbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:10.290045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407c527-9101-4d72-a76b-d8fc718fbbf5 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:10.290635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407c527-9101-4d72-a76b-d8fc718fbbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:10.290859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407c527-9101-4d72-a76b-d8fc718fbbf5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:10.298693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421965c-e393-4f1a-a2fe-5f332cfc3427 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:10.304889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407c527-9101-4d72-a76b-d8fc718fbbf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-e407c527-9101-4d72-a76b-d8fc718fbbf5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:10.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:10.313673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:10.314489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:10.314489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:10.314489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:10.314489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:10.314489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:10.314489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:10.314489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:10.314489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:10.314489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:10.314489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:10.314489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:23:10.314489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:10.314489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 173/357] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:10.316241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5c284-012c-4988-8a8e-83db9aa5fc65 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:10.316576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5c284-012c-4988-8a8e-83db9aa5fc65 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:10.316908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5c284-012c-4988-8a8e-83db9aa5fc65 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:10.319773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 174/358] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:10.324813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5c284-012c-4988-8a8e-83db9aa5fc65 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:10.351432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f89c2-b240-4d53-b659-bec4f8802dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:10.352421 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f89c2-b240-4d53-b659-bec4f8802dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:10.352757 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f89c2-b240-4d53-b659-bec4f8802dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:10.353244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f89c2-b240-4d53-b659-bec4f8802dc0 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:10.355248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f89c2-b240-4d53-b659-bec4f8802dc0 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:10.357158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f89c2-b240-4d53-b659-bec4f8802dc0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:10.360075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 185/359] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:10 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:10.361308 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f89c2-b240-4d53-b659-bec4f8802dc0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:10.381751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:10.451045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5c284-012c-4988-8a8e-83db9aa5fc65 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:10.463621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:10.495803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfe45d-1037-40a5-bf0a-39377cdfd486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:10.496507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfe45d-1037-40a5-bf0a-39377cdfd486 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:10.496705 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfe45d-1037-40a5-bf0a-39377cdfd486 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:10.497148 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfe45d-1037-40a5-bf0a-39377cdfd486 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:10.498062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdfe45d-1037-40a5-bf0a-39377cdfd486 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:10.499497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdfe45d-1037-40a5-bf0a-39377cdfd486 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:10.500323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f89c2-b240-4d53-b659-bec4f8802dc0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:10.501479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 175/360] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:10 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:10.502630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bd2ef-2265-4e90-9067-643320c61100 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:10.502901 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bd2ef-2265-4e90-9067-643320c61100 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:10.503163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bd2ef-2265-4e90-9067-643320c61100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:10.524718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-7c0bd2ef-2265-4e90-9067-643320c61100 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:10.526044 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7c0bd2ef-2265-4e90-9067-643320c61100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:10.544725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445c45c-8dd2-4a00-94f8-14a2383272fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:10.545347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445c45c-8dd2-4a00-94f8-14a2383272fb None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:10.545924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445c45c-8dd2-4a00-94f8-14a2383272fb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:10.545924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445c45c-8dd2-4a00-94f8-14a2383272fb None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:10.546541 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445c45c-8dd2-4a00-94f8-14a2383272fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:10.546843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445c45c-8dd2-4a00-94f8-14a2383272fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:10.548250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445c45c-8dd2-4a00-94f8-14a2383272fb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:10.557046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 186/361] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:23:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:10.557902 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445c45c-8dd2-4a00-94f8-14a2383272fb None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:10.605061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 176/362] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:10.605803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bd2ef-2265-4e90-9067-643320c61100 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:10.654264 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:10.694865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445c45c-8dd2-4a00-94f8-14a2383272fb None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:10.715579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bd2ef-2265-4e90-9067-643320c61100 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:10.739256 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dc395-cda2-4a28-a02d-826c81b0333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:10.740514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dc395-cda2-4a28-a02d-826c81b0333b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:10.741224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dc395-cda2-4a28-a02d-826c81b0333b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:10.741682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dc395-cda2-4a28-a02d-826c81b0333b None admin] PATH_INFO: `/v3/groups/459bcbad31f44b8fad10cdd5de14856c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:10.743799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1dc395-cda2-4a28-a02d-826c81b0333b None admin] RBAC: Authorizing `identity:get_group(group_id=459bcbad31f44b8fad10cdd5de14856c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:10.746649 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1dc395-cda2-4a28-a02d-826c81b0333b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:10.751263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 187/363] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:10 2022] GET /identity/v3/groups/459bcbad31f44b8fad10cdd5de14856c => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:10.751263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dc395-cda2-4a28-a02d-826c81b0333b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:10.763774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:10.769364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782fe36-cdab-4474-9ad3-5aac48d525a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:10.769364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782fe36-cdab-4474-9ad3-5aac48d525a3 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:10.769364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782fe36-cdab-4474-9ad3-5aac48d525a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:10.769640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782fe36-cdab-4474-9ad3-5aac48d525a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:10.788176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782fe36-cdab-4474-9ad3-5aac48d525a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-f782fe36-cdab-4474-9ad3-5aac48d525a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:10.794779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:10.795713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:10.797346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:10.797346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:10.797346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:10.797346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:10.797346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:10.797346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:10.797346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:10.797346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:10.797346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:10.797346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:10.797346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:23:10.797346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:10.797346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 177/364] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:23:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:10.798317 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782fe36-cdab-4474-9ad3-5aac48d525a3 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:10.892816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dc395-cda2-4a28-a02d-826c81b0333b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:10.932288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54c815-e62f-4eec-82ac-0549311974a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:10.932929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54c815-e62f-4eec-82ac-0549311974a2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:10.933202 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54c815-e62f-4eec-82ac-0549311974a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:10.933468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54c815-e62f-4eec-82ac-0549311974a2 None admin] PATH_INFO: `/v3/groups/459bcbad31f44b8fad10cdd5de14856c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:10.934584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54c815-e62f-4eec-82ac-0549311974a2 None admin] RBAC: Authorizing `identity:get_group(group_id=459bcbad31f44b8fad10cdd5de14856c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:10.934996 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782fe36-cdab-4474-9ad3-5aac48d525a3 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:10.936045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54c815-e62f-4eec-82ac-0549311974a2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:10.937896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:10.938005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 188/365] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:10 2022] GET /identity/v3/groups/459bcbad31f44b8fad10cdd5de14856c => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:10.939293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54c815-e62f-4eec-82ac-0549311974a2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:10.975210 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c9bfb-019b-4ebc-8e2f-d5dae66cec7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:10.976247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c9bfb-019b-4ebc-8e2f-d5dae66cec7d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:10.976711 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c9bfb-019b-4ebc-8e2f-d5dae66cec7d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:10.977162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c9bfb-019b-4ebc-8e2f-d5dae66cec7d None admin] PATH_INFO: `/v3/users/90b7de9cd6e64ec7a293d478020a0af3` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:10.978848 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c9bfb-019b-4ebc-8e2f-d5dae66cec7d None admin] RBAC: Authorizing `identity:get_user(user_id=90b7de9cd6e64ec7a293d478020a0af3)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:10.980538 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c9bfb-019b-4ebc-8e2f-d5dae66cec7d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:10.982266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 178/366] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:23:10 2022] GET /identity/v3/users/90b7de9cd6e64ec7a293d478020a0af3 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:10.984835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c9bfb-019b-4ebc-8e2f-d5dae66cec7d None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:11.023353 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:11.057105 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54c815-e62f-4eec-82ac-0549311974a2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:11.100359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c15d3-481f-47cd-ac7e-e8d36c4b9bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:11.100977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c15d3-481f-47cd-ac7e-e8d36c4b9bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.101250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c15d3-481f-47cd-ac7e-e8d36c4b9bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.101518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c15d3-481f-47cd-ac7e-e8d36c4b9bc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.101800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c9bfb-019b-4ebc-8e2f-d5dae66cec7d None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:11.102239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c15d3-481f-47cd-ac7e-e8d36c4b9bc3 None admin] RBAC: Adding query filter params (group.id=459bcbad31f44b8fad10cdd5de14856c, role.id=be3ffa4db9954414868f116f97f3a929, scope.project.id=ce0bf520fc554803bd196782f428d532) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:11.102540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c15d3-481f-47cd-ac7e-e8d36c4b9bc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:11.105703 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c15d3-481f-47cd-ac7e-e8d36c4b9bc3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:11.118002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 189/367] 10.222.0.62 () {56 vars in 1376 bytes} [Thu Sep 1 16:23:10 2022] GET /identity/v3/role_assignments?group.id=459bcbad31f44b8fad10cdd5de14856c&scope.project.id=ce0bf520fc554803bd196782f428d532&role.id=be3ffa4db9954414868f116f97f3a929 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:11.119302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c15d3-481f-47cd-ac7e-e8d36c4b9bc3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:11.146925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734749da-e3ca-4d95-93bf-f95392dc52ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:11.147617 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734749da-e3ca-4d95-93bf-f95392dc52ff None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.147938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734749da-e3ca-4d95-93bf-f95392dc52ff None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.148259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734749da-e3ca-4d95-93bf-f95392dc52ff None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.148965 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734749da-e3ca-4d95-93bf-f95392dc52ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:11.149270 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734749da-e3ca-4d95-93bf-f95392dc52ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:11.151290 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734749da-e3ca-4d95-93bf-f95392dc52ff None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:11.163810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 179/368] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:11.168431 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734749da-e3ca-4d95-93bf-f95392dc52ff None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:11.216943 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:11.276481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c15d3-481f-47cd-ac7e-e8d36c4b9bc3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:11.301688 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734749da-e3ca-4d95-93bf-f95392dc52ff None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:11.324881 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80534c37-73da-4c35-8bc4-04566456d605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:11.325541 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80534c37-73da-4c35-8bc4-04566456d605 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.325845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80534c37-73da-4c35-8bc4-04566456d605 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.326079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80534c37-73da-4c35-8bc4-04566456d605 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.331681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:11.344517 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80534c37-73da-4c35-8bc4-04566456d605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:11.346295 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7e9f6-1d64-4bff-9432-18441ce1cf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:11.346987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7e9f6-1d64-4bff-9432-18441ce1cf1a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.347334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7e9f6-1d64-4bff-9432-18441ce1cf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.347675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7e9f6-1d64-4bff-9432-18441ce1cf1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.348502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7e9f6-1d64-4bff-9432-18441ce1cf1a None admin] RBAC: Adding query filter params (group.id=459bcbad31f44b8fad10cdd5de14856c, role.id=3100f6054f7d4d7b984fe19a4d8ba291, scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:11.348875 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7e9f6-1d64-4bff-9432-18441ce1cf1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:11.350434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7e9f6-1d64-4bff-9432-18441ce1cf1a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-80534c37-73da-4c35-8bc4-04566456d605 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:11.352590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:11.353642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:11.354557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:11.354557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:11.354557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:11.354557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:11.354557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:11.354557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:11.354557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:11.354557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:11.354557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:11.354557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:11.354557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:11.354557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:11.357531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 190/369] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:23:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:11.358773 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80534c37-73da-4c35-8bc4-04566456d605 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:11.359795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 180/370] 10.222.0.62 () {56 vars in 1376 bytes} [Thu Sep 1 16:23:11 2022] GET /identity/v3/role_assignments?group.id=459bcbad31f44b8fad10cdd5de14856c&scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6&role.id=3100f6054f7d4d7b984fe19a4d8ba291 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:11.360656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7e9f6-1d64-4bff-9432-18441ce1cf1a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:11.473540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80534c37-73da-4c35-8bc4-04566456d605 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:11.530358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47eae9-c232-4a34-a5a1-0167e098d3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:11.531371 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47eae9-c232-4a34-a5a1-0167e098d3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.531869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47eae9-c232-4a34-a5a1-0167e098d3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.532375 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47eae9-c232-4a34-a5a1-0167e098d3f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.532516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:11.533843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47eae9-c232-4a34-a5a1-0167e098d3f2 None admin] RBAC: Adding query filter params (role.id=16095b51965d46fe87bf7fc60e42bdb0, user.id=90b7de9cd6e64ec7a293d478020a0af3, scope.system=all) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:11.534266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47eae9-c232-4a34-a5a1-0167e098d3f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:11.535933 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47eae9-c232-4a34-a5a1-0167e098d3f2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:11.538488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7e9f6-1d64-4bff-9432-18441ce1cf1a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:11.566571 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 191/371] 10.222.0.62 () {56 vars in 1308 bytes} [Thu Sep 1 16:23:11 2022] GET /identity/v3/role_assignments?user.id=90b7de9cd6e64ec7a293d478020a0af3&scope.system=all&role.id=16095b51965d46fe87bf7fc60e42bdb0 => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:11.567604 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47eae9-c232-4a34-a5a1-0167e098d3f2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:11.603949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b11778-c6f8-42af-a439-1577e6b0ea3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:11.604899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b11778-c6f8-42af-a439-1577e6b0ea3d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.605347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b11778-c6f8-42af-a439-1577e6b0ea3d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.605746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b11778-c6f8-42af-a439-1577e6b0ea3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.693683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:11.733011 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b11778-c6f8-42af-a439-1577e6b0ea3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:11.734695 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b11778-c6f8-42af-a439-1577e6b0ea3d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:11.750657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47eae9-c232-4a34-a5a1-0167e098d3f2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:11.790207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6b868-aaf3-4775-a4c4-7f571f489dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:11.790898 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6b868-aaf3-4775-a4c4-7f571f489dfc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.791174 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6b868-aaf3-4775-a4c4-7f571f489dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.791445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6b868-aaf3-4775-a4c4-7f571f489dfc None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.792148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6b868-aaf3-4775-a4c4-7f571f489dfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:11.792509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6b868-aaf3-4775-a4c4-7f571f489dfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:11.793897 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6b868-aaf3-4775-a4c4-7f571f489dfc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:11.803466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 192/372] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:11.809432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6b868-aaf3-4775-a4c4-7f571f489dfc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:11.922636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6b868-aaf3-4775-a4c4-7f571f489dfc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:11.938015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 181/373] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:11.939325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d82bf3-d72f-435e-b94e-60be2d8f33f0 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.939608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d82bf3-d72f-435e-b94e-60be2d8f33f0 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.939897 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d82bf3-d72f-435e-b94e-60be2d8f33f0 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.944211 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:11.945357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 182/374] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:11.946703 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:11.947408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb7be8-6b53-4bbe-bae3-3dc203383347 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.947864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb7be8-6b53-4bbe-bae3-3dc203383347 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.948196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb7be8-6b53-4bbe-bae3-3dc203383347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.968610 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec855df0-4a1d-4e61-98c6-7f3ffa1be501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:11.969234 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec855df0-4a1d-4e61-98c6-7f3ffa1be501 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:11.969512 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec855df0-4a1d-4e61-98c6-7f3ffa1be501 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:11.969786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec855df0-4a1d-4e61-98c6-7f3ffa1be501 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:11.982638 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-50cb7be8-6b53-4bbe-bae3-3dc203383347 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:11.983995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-50cb7be8-6b53-4bbe-bae3-3dc203383347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:12.052634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec855df0-4a1d-4e61-98c6-7f3ffa1be501 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:12.054602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec855df0-4a1d-4e61-98c6-7f3ffa1be501 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:12.082450 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 183/375] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:23:12.083322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb7be8-6b53-4bbe-bae3-3dc203383347 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:12.185069 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 193/376] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:23:12.191086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec855df0-4a1d-4e61-98c6-7f3ffa1be501 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:12.207583 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb7be8-6b53-4bbe-bae3-3dc203383347 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:12.214463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:12.247166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:12.259193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e87045-eded-4bea-a20a-6c6b29c93529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:12.260187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e87045-eded-4bea-a20a-6c6b29c93529 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:12.260474 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e87045-eded-4bea-a20a-6c6b29c93529 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:12.260752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e87045-eded-4bea-a20a-6c6b29c93529 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:12.262294 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e87045-eded-4bea-a20a-6c6b29c93529 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:12.264929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e87045-eded-4bea-a20a-6c6b29c93529 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:12.267327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 184/377] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:12 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:12.273608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e87045-eded-4bea-a20a-6c6b29c93529 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:12.346583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec855df0-4a1d-4e61-98c6-7f3ffa1be501 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:12.389356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e87045-eded-4bea-a20a-6c6b29c93529 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:12.409545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2184bef-50ba-400d-8a4f-496209176a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:12.410249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2184bef-50ba-400d-8a4f-496209176a09 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:12.410461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2184bef-50ba-400d-8a4f-496209176a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:12.410726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2184bef-50ba-400d-8a4f-496209176a09 None admin] PATH_INFO: `/v3/projects/026b4ed9c3cf48a48d86818cba23c13c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:12.412570 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2184bef-50ba-400d-8a4f-496209176a09 None admin] RBAC: Authorizing `identity:get_project(project_id=026b4ed9c3cf48a48d86818cba23c13c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:12.414029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2184bef-50ba-400d-8a4f-496209176a09 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:12.416311 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 194/378] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:12 2022] GET /identity/v3/projects/026b4ed9c3cf48a48d86818cba23c13c => generated 344 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:12.428885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2184bef-50ba-400d-8a4f-496209176a09 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:12.432972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539e631-2258-44a7-9821-f51d6861f3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:12.433626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539e631-2258-44a7-9821-f51d6861f3a3 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:12.433899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539e631-2258-44a7-9821-f51d6861f3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:12.434492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539e631-2258-44a7-9821-f51d6861f3a3 None admin] PATH_INFO: `/v3/groups/459bcbad31f44b8fad10cdd5de14856c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:12.435617 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539e631-2258-44a7-9821-f51d6861f3a3 None admin] RBAC: Authorizing `identity:get_group(group_id=459bcbad31f44b8fad10cdd5de14856c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:12.437203 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539e631-2258-44a7-9821-f51d6861f3a3 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:12.439378 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 185/379] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:12 2022] GET /identity/v3/groups/459bcbad31f44b8fad10cdd5de14856c => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:12.445076 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539e631-2258-44a7-9821-f51d6861f3a3 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:12.528407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:12.545430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2184bef-50ba-400d-8a4f-496209176a09 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:12.560319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:12.599658 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea710e0-0abf-4d15-9474-22157cd438a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:12.600365 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea710e0-0abf-4d15-9474-22157cd438a9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:12.600603 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea710e0-0abf-4d15-9474-22157cd438a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:12.600876 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea710e0-0abf-4d15-9474-22157cd438a9 None admin] PATH_INFO: `/v3/groups/6670d385a6164f81a07228147028d866` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:12.613630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea710e0-0abf-4d15-9474-22157cd438a9 None admin] RBAC: Authorizing `identity:get_group(group_id=6670d385a6164f81a07228147028d866)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:12.615366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea710e0-0abf-4d15-9474-22157cd438a9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:12.619916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 195/380] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:12 2022] GET /identity/v3/groups/6670d385a6164f81a07228147028d866 => generated 229 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:12.623115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539e631-2258-44a7-9821-f51d6861f3a3 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:12.626462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea710e0-0abf-4d15-9474-22157cd438a9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:12.690336 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30e5a3-a47b-49fd-841b-760ee14e92d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:12.690952 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30e5a3-a47b-49fd-841b-760ee14e92d4 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:12.691229 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30e5a3-a47b-49fd-841b-760ee14e92d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:12.691498 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30e5a3-a47b-49fd-841b-760ee14e92d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:12.692908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30e5a3-a47b-49fd-841b-760ee14e92d4 None admin] RBAC: Adding query filter params (group.id=459bcbad31f44b8fad10cdd5de14856c, role.id=be3ffa4db9954414868f116f97f3a929, scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:12.693222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30e5a3-a47b-49fd-841b-760ee14e92d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:12.695624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30e5a3-a47b-49fd-841b-760ee14e92d4 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:12.704268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 186/381] 10.222.0.62 () {56 vars in 1376 bytes} [Thu Sep 1 16:23:12 2022] GET /identity/v3/role_assignments?group.id=459bcbad31f44b8fad10cdd5de14856c&scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6&role.id=be3ffa4db9954414868f116f97f3a929 => generated 260 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:12.705696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30e5a3-a47b-49fd-841b-760ee14e92d4 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:12.771526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea710e0-0abf-4d15-9474-22157cd438a9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:12.813225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4d4ad-4993-4282-b2aa-b827792f861a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:12.813907 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4d4ad-4993-4282-b2aa-b827792f861a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:12.814187 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4d4ad-4993-4282-b2aa-b827792f861a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:12.814510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4d4ad-4993-4282-b2aa-b827792f861a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:12.815706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4d4ad-4993-4282-b2aa-b827792f861a None admin] RBAC: Adding query filter params (group.id=6670d385a6164f81a07228147028d866, role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=026b4ed9c3cf48a48d86818cba23c13c) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:12.816194 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4d4ad-4993-4282-b2aa-b827792f861a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:12.817812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4d4ad-4993-4282-b2aa-b827792f861a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:12.825974 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 196/382] 10.222.0.62 () {56 vars in 1376 bytes} [Thu Sep 1 16:23:12 2022] GET /identity/v3/role_assignments?group.id=6670d385a6164f81a07228147028d866&scope.project.id=026b4ed9c3cf48a48d86818cba23c13c&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:12.881782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30e5a3-a47b-49fd-841b-760ee14e92d4 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:12.924355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:12.934021 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e3597-56b7-4b21-8145-98bf5c8a959f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:12.934687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e3597-56b7-4b21-8145-98bf5c8a959f None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:12.934964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e3597-56b7-4b21-8145-98bf5c8a959f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:12.935235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e3597-56b7-4b21-8145-98bf5c8a959f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:12.948433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e3597-56b7-4b21-8145-98bf5c8a959f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-873e3597-56b7-4b21-8145-98bf5c8a959f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:12.956938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:12.957844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:12.958609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:12.958609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:12.958609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:12.958609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:12.958609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:12.958609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:12.958609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:12.958609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:12.958609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:12.958609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:12.958609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:12.958609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:12.959202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 187/383] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:12.967853 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4d4ad-4993-4282-b2aa-b827792f861a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:12.992912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:13.099057 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4d4ad-4993-4282-b2aa-b827792f861a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:13.136782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489749a4-ea8a-4721-aa1f-24044999b8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:13.137330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489749a4-ea8a-4721-aa1f-24044999b8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:13.137608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489749a4-ea8a-4721-aa1f-24044999b8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:13.137879 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489749a4-ea8a-4721-aa1f-24044999b8e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:13.138544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489749a4-ea8a-4721-aa1f-24044999b8e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:13.138844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489749a4-ea8a-4721-aa1f-24044999b8e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:13.140254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489749a4-ea8a-4721-aa1f-24044999b8e1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:13.149654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 197/384] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:23:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:13.155814 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e3597-56b7-4b21-8145-98bf5c8a959f None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:13.262975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e3597-56b7-4b21-8145-98bf5c8a959f None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:13.301306 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9744d5-19c4-43d0-aa20-6cae5512184a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:13.302279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9744d5-19c4-43d0-aa20-6cae5512184a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:13.302693 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9744d5-19c4-43d0-aa20-6cae5512184a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:13.303076 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9744d5-19c4-43d0-aa20-6cae5512184a None admin] PATH_INFO: `/v3/groups/459bcbad31f44b8fad10cdd5de14856c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:13.304734 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9744d5-19c4-43d0-aa20-6cae5512184a None admin] RBAC: Authorizing `identity:get_group(group_id=459bcbad31f44b8fad10cdd5de14856c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:13.307239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9744d5-19c4-43d0-aa20-6cae5512184a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:13.310208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 188/385] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:13 2022] GET /identity/v3/groups/459bcbad31f44b8fad10cdd5de14856c => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:13.318327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489749a4-ea8a-4721-aa1f-24044999b8e1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:13.437728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489749a4-ea8a-4721-aa1f-24044999b8e1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:13.440188 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:13.475579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b989b-2db9-4143-a1fa-7006e9804d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:13.476264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b989b-2db9-4143-a1fa-7006e9804d8c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:13.476543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b989b-2db9-4143-a1fa-7006e9804d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:13.476812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b989b-2db9-4143-a1fa-7006e9804d8c None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:13.478285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b989b-2db9-4143-a1fa-7006e9804d8c None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:13.479927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b989b-2db9-4143-a1fa-7006e9804d8c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:13.481662 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 198/386] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:13 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:13.487504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9744d5-19c4-43d0-aa20-6cae5512184a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:13.557926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:13.610786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9744d5-19c4-43d0-aa20-6cae5512184a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:13.653299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafac68-edc5-4410-9b9a-0b30b9b57b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:13.654356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafac68-edc5-4410-9b9a-0b30b9b57b91 None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:13.654857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafac68-edc5-4410-9b9a-0b30b9b57b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:13.655328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafac68-edc5-4410-9b9a-0b30b9b57b91 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532/groups/459bcbad31f44b8fad10cdd5de14856c/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:13.660827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafac68-edc5-4410-9b9a-0b30b9b57b91 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce0bf520fc554803bd196782f428d532, group_id=459bcbad31f44b8fad10cdd5de14856c, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:13.662503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafac68-edc5-4410-9b9a-0b30b9b57b91 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:13.690396 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 189/387] 10.222.0.62 () {58 vars in 1442 bytes} [Thu Sep 1 16:23:13 2022] PUT /identity/v3/projects/ce0bf520fc554803bd196782f428d532/groups/459bcbad31f44b8fad10cdd5de14856c/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:23:13.999513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cde03d-fe86-43e7-adc9-f0caeca40ced None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:14.000037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cde03d-fe86-43e7-adc9-f0caeca40ced None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:14.000037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cde03d-fe86-43e7-adc9-f0caeca40ced None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:14.001627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 199/388] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:14.006844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb70d3a-306d-4133-a858-0a2acceb841f None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:14.007367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb70d3a-306d-4133-a858-0a2acceb841f None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:14.007830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb70d3a-306d-4133-a858-0a2acceb841f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:14.028450 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-4eb70d3a-306d-4133-a858-0a2acceb841f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:14.029968 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-4eb70d3a-306d-4133-a858-0a2acceb841f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:14.068939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e8072-c77d-4af7-b58a-44e707bc5ebc None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:14.069537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e8072-c77d-4af7-b58a-44e707bc5ebc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:14.069996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e8072-c77d-4af7-b58a-44e707bc5ebc None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:14.074502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 200/389] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:14.090088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e443f-87d7-46e5-ac67-ae0a039de548 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:14.090716 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e443f-87d7-46e5-ac67-ae0a039de548 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:14.091175 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e443f-87d7-46e5-ac67-ae0a039de548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:14.110577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 190/390] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:23:14.121772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-509e443f-87d7-46e5-ac67-ae0a039de548 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:14.123229 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-509e443f-87d7-46e5-ac67-ae0a039de548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:14.213762 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 201/391] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:14.486502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb70d3a-306d-4133-a858-0a2acceb841f None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:14.548071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:14.582055 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e443f-87d7-46e5-ac67-ae0a039de548 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:14.608334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb70d3a-306d-4133-a858-0a2acceb841f None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:14.648525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bca977-feab-481c-9300-a697566fde74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:14.650159 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bca977-feab-481c-9300-a697566fde74 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:14.651040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bca977-feab-481c-9300-a697566fde74 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:14.652172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bca977-feab-481c-9300-a697566fde74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:14.677360 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bca977-feab-481c-9300-a697566fde74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-24bca977-feab-481c-9300-a697566fde74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:14.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:14.685835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:14.686685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:14.686685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:14.686685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:14.686685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:14.686685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:14.686685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:14.686685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:14.686685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:14.686685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:14.686685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:14.686685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:14.686685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:14.690998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 191/392] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:14.697831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bca977-feab-481c-9300-a697566fde74 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:14.732552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:14.768578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e443f-87d7-46e5-ac67-ae0a039de548 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:14.812639 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1a4f9-1b8b-45fd-800d-2ecf2892ba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:14.813291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a4f9-1b8b-45fd-800d-2ecf2892ba4b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:14.813561 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a4f9-1b8b-45fd-800d-2ecf2892ba4b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:14.813823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a4f9-1b8b-45fd-800d-2ecf2892ba4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:14.833842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1a4f9-1b8b-45fd-800d-2ecf2892ba4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b4f1a4f9-1b8b-45fd-800d-2ecf2892ba4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:14.838789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:14.839822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:14.840771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:14.840771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:14.840771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:14.840771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:14.840771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:14.840771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:14.840771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:14.840771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:14.840771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:14.840771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:14.840771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:23:14.840771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:14.840771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 202/393] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:23:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:14.850490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1a4f9-1b8b-45fd-800d-2ecf2892ba4b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:14.917258 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bca977-feab-481c-9300-a697566fde74 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:14.940277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:14.985695 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85937cba-388f-4362-a2df-bf4926535286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:14.986332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85937cba-388f-4362-a2df-bf4926535286 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:14.986607 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85937cba-388f-4362-a2df-bf4926535286 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:14.986872 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85937cba-388f-4362-a2df-bf4926535286 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:14.987641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85937cba-388f-4362-a2df-bf4926535286 None admin] RBAC: Adding query filter params (name=member) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:14.987972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85937cba-388f-4362-a2df-bf4926535286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:14.989480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85937cba-388f-4362-a2df-bf4926535286 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:15.000339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 192/394] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:23:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:15.001896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8193f-1791-4736-abf5-99281d3919f3 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.002181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8193f-1791-4736-abf5-99281d3919f3 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.002451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8193f-1791-4736-abf5-99281d3919f3 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.004616 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1a4f9-1b8b-45fd-800d-2ecf2892ba4b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:15.008037 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 193/395] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:15.008949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8193f-1791-4736-abf5-99281d3919f3 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:15.036529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:15.042136 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4e86d-c5c0-41c1-8c32-2424680b19c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:15.043145 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4e86d-c5c0-41c1-8c32-2424680b19c3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.043602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4e86d-c5c0-41c1-8c32-2424680b19c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.044084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4e86d-c5c0-41c1-8c32-2424680b19c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.044934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4e86d-c5c0-41c1-8c32-2424680b19c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:15.045399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4e86d-c5c0-41c1-8c32-2424680b19c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:15.047025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4e86d-c5c0-41c1-8c32-2424680b19c3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:15.056343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 203/396] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:15.058047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1eeaf-7281-4bef-8482-d0b36add5818 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.058715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1eeaf-7281-4bef-8482-d0b36add5818 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.059160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1eeaf-7281-4bef-8482-d0b36add5818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.078385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ebb1eeaf-7281-4bef-8482-d0b36add5818 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:15.079901 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ebb1eeaf-7281-4bef-8482-d0b36add5818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:15.128566 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8193f-1791-4736-abf5-99281d3919f3 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:15.152593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 204/397] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:15.153787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1eeaf-7281-4bef-8482-d0b36add5818 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:15.169136 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34106270-d5d2-4fe2-bb35-f262ee3ec26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:15.169856 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34106270-d5d2-4fe2-bb35-f262ee3ec26b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.170052 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34106270-d5d2-4fe2-bb35-f262ee3ec26b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.170310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34106270-d5d2-4fe2-bb35-f262ee3ec26b None admin] PATH_INFO: `/v3/groups/459bcbad31f44b8fad10cdd5de14856c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.176721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34106270-d5d2-4fe2-bb35-f262ee3ec26b None admin] RBAC: Authorizing `identity:get_group(group_id=459bcbad31f44b8fad10cdd5de14856c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:15.178380 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34106270-d5d2-4fe2-bb35-f262ee3ec26b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:15.179856 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 194/398] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:15 2022] GET /identity/v3/groups/459bcbad31f44b8fad10cdd5de14856c => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:15.185899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34106270-d5d2-4fe2-bb35-f262ee3ec26b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:15.262368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:15.303401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1eeaf-7281-4bef-8482-d0b36add5818 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:15.349372 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:15.373164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea897139-3408-4d2f-abea-4536140e2994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:15.373547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34106270-d5d2-4fe2-bb35-f262ee3ec26b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:15.374905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea897139-3408-4d2f-abea-4536140e2994 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.375740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea897139-3408-4d2f-abea-4536140e2994 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.376605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea897139-3408-4d2f-abea-4536140e2994 None admin] PATH_INFO: `/v3/groups/459bcbad31f44b8fad10cdd5de14856c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.381775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea897139-3408-4d2f-abea-4536140e2994 None admin] RBAC: Authorizing `identity:get_group(group_id=459bcbad31f44b8fad10cdd5de14856c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:15.383646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea897139-3408-4d2f-abea-4536140e2994 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:15.385979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 205/399] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:15 2022] GET /identity/v3/groups/459bcbad31f44b8fad10cdd5de14856c => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:15.393196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea897139-3408-4d2f-abea-4536140e2994 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:15.412222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282fc963-d8cc-4e14-b031-36104d4f7b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:15.412872 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fc963-d8cc-4e14-b031-36104d4f7b63 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.413191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fc963-d8cc-4e14-b031-36104d4f7b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.413465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fc963-d8cc-4e14-b031-36104d4f7b63 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.414681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282fc963-d8cc-4e14-b031-36104d4f7b63 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:15.417169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282fc963-d8cc-4e14-b031-36104d4f7b63 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:15.419615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 195/400] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:15 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:15.425050 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282fc963-d8cc-4e14-b031-36104d4f7b63 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:15.544176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282fc963-d8cc-4e14-b031-36104d4f7b63 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:15.576282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea897139-3408-4d2f-abea-4536140e2994 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:15.577032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:15.587391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b410a83-c684-4cc8-b670-4bcd5b49a748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:15.588356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b410a83-c684-4cc8-b670-4bcd5b49a748 None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.589419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b410a83-c684-4cc8-b670-4bcd5b49a748 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.589770 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b410a83-c684-4cc8-b670-4bcd5b49a748 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6/groups/459bcbad31f44b8fad10cdd5de14856c/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.592139 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b410a83-c684-4cc8-b670-4bcd5b49a748 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ab355ce83f94db6a7c315ba27ae52a6, group_id=459bcbad31f44b8fad10cdd5de14856c, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:15.594214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b410a83-c684-4cc8-b670-4bcd5b49a748 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:15.615525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 196/401] 10.222.0.62 () {58 vars in 1442 bytes} [Thu Sep 1 16:23:15 2022] PUT /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6/groups/459bcbad31f44b8fad10cdd5de14856c/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:23:15.616896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b410a83-c684-4cc8-b670-4bcd5b49a748 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:15.617964 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ce120-a5b7-4cbf-b8f5-b90242982058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:15.618720 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ce120-a5b7-4cbf-b8f5-b90242982058 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.618997 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ce120-a5b7-4cbf-b8f5-b90242982058 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.619414 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ce120-a5b7-4cbf-b8f5-b90242982058 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.620969 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ce120-a5b7-4cbf-b8f5-b90242982058 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:15.622584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ce120-a5b7-4cbf-b8f5-b90242982058 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:15.626014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 206/402] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:15 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:15.632454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ce120-a5b7-4cbf-b8f5-b90242982058 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:15.718420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:15.747325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b410a83-c684-4cc8-b670-4bcd5b49a748 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:15.769681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ce120-a5b7-4cbf-b8f5-b90242982058 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:15.796990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9197c9-808a-4399-b4bd-dc7ff7a00066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:15.798444 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9197c9-808a-4399-b4bd-dc7ff7a00066 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.798944 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9197c9-808a-4399-b4bd-dc7ff7a00066 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.799376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9197c9-808a-4399-b4bd-dc7ff7a00066 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.810832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5c050-4b7a-48b4-a15d-2399a0b3fdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:15.811525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5c050-4b7a-48b4-a15d-2399a0b3fdb2 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.811802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5c050-4b7a-48b4-a15d-2399a0b3fdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.812129 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5c050-4b7a-48b4-a15d-2399a0b3fdb2 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532/groups/459bcbad31f44b8fad10cdd5de14856c/roles/be3ffa4db9954414868f116f97f3a929` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.813025 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9197c9-808a-4399-b4bd-dc7ff7a00066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:15.814408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5c050-4b7a-48b4-a15d-2399a0b3fdb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce0bf520fc554803bd196782f428d532, group_id=459bcbad31f44b8fad10cdd5de14856c, role_id=be3ffa4db9954414868f116f97f3a929)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:15.815922 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5c050-4b7a-48b4-a15d-2399a0b3fdb2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-5c9197c9-808a-4399-b4bd-dc7ff7a00066 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:15.819782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:15.820876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:15.821852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:15.821852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:15.821852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:15.821852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:15.821852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:15.821852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:15.821852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:15.821852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:15.821852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:15.821852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:15.821852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:23:15.821852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:15.824246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 197/403] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:23:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:15.825793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190df93-9ad4-4d63-ac92-de23625cfd21 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.826375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190df93-9ad4-4d63-ac92-de23625cfd21 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.826867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190df93-9ad4-4d63-ac92-de23625cfd21 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.828814 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 198/404] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:15.830434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190df93-9ad4-4d63-ac92-de23625cfd21 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:15.837306 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 207/405] 10.222.0.62 () {58 vars in 1442 bytes} [Thu Sep 1 16:23:15 2022] PUT /identity/v3/projects/ce0bf520fc554803bd196782f428d532/groups/459bcbad31f44b8fad10cdd5de14856c/roles/be3ffa4db9954414868f116f97f3a929 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:23:15.842380 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae3d40-f190-4e74-8548-ed50ae8706b8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:15.842380 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae3d40-f190-4e74-8548-ed50ae8706b8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:15.842380 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae3d40-f190-4e74-8548-ed50ae8706b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:15.865329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bdae3d40-f190-4e74-8548-ed50ae8706b8 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:15.866571 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bdae3d40-f190-4e74-8548-ed50ae8706b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:15.870368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:15.961425 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 208/406] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:15.964736 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190df93-9ad4-4d63-ac92-de23625cfd21 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:16.013307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddd46a-ada9-4cc9-a30c-284f1410b311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:16.014379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddd46a-ada9-4cc9-a30c-284f1410b311 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:16.014845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddd46a-ada9-4cc9-a30c-284f1410b311 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:16.015309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddd46a-ada9-4cc9-a30c-284f1410b311 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:16.016203 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddd46a-ada9-4cc9-a30c-284f1410b311 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:16.016699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddd46a-ada9-4cc9-a30c-284f1410b311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:16.018319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddd46a-ada9-4cc9-a30c-284f1410b311 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:16.033945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 199/407] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:16.039636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae3d40-f190-4e74-8548-ed50ae8706b8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:16.105198 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:16.157993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae3d40-f190-4e74-8548-ed50ae8706b8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:16.207719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334463d-7fd5-4290-a9e4-f0168bdda861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:16.208395 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334463d-7fd5-4290-a9e4-f0168bdda861 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:16.208673 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334463d-7fd5-4290-a9e4-f0168bdda861 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:16.208945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334463d-7fd5-4290-a9e4-f0168bdda861 None admin] PATH_INFO: `/v3/groups/459bcbad31f44b8fad10cdd5de14856c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:16.210060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334463d-7fd5-4290-a9e4-f0168bdda861 None admin] RBAC: Authorizing `identity:get_group(group_id=459bcbad31f44b8fad10cdd5de14856c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:16.211554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334463d-7fd5-4290-a9e4-f0168bdda861 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:16.213823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 209/408] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:16 2022] GET /identity/v3/groups/459bcbad31f44b8fad10cdd5de14856c => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:16.219201 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddd46a-ada9-4cc9-a30c-284f1410b311 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:16.249896 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334463d-7fd5-4290-a9e4-f0168bdda861 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:16.354140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:16.370957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddd46a-ada9-4cc9-a30c-284f1410b311 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:16.385328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334463d-7fd5-4290-a9e4-f0168bdda861 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:16.407662 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e528bc-bcab-4ee2-bb35-4ca44f372ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:16.408515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e528bc-bcab-4ee2-bb35-4ca44f372ffe None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:16.408814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:16.408952 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e528bc-bcab-4ee2-bb35-4ca44f372ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:16.409325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e528bc-bcab-4ee2-bb35-4ca44f372ffe None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:16.410812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e528bc-bcab-4ee2-bb35-4ca44f372ffe None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:16.412552 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e528bc-bcab-4ee2-bb35-4ca44f372ffe None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:16.414909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 200/409] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:16 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:16.420771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7be8c-774b-429c-bd24-56855916e52e None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:16.421175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7be8c-774b-429c-bd24-56855916e52e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:16.421548 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7be8c-774b-429c-bd24-56855916e52e None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:16.423293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 201/410] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:16.425154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7be8c-774b-429c-bd24-56855916e52e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:16.428559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a895759-6d6b-47a4-8632-7c51b17a4fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:16.430145 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a895759-6d6b-47a4-8632-7c51b17a4fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:16.430429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a895759-6d6b-47a4-8632-7c51b17a4fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:16.430700 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a895759-6d6b-47a4-8632-7c51b17a4fa3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:16.442939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a895759-6d6b-47a4-8632-7c51b17a4fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8a895759-6d6b-47a4-8632-7c51b17a4fa3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:16.450249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:16.451683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:16.452678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:16.452678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:16.452678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:16.452678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:16.452678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:16.452678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:16.452678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:16.452678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:16.452678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:16.452678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:16.452678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:16.452678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:16.455971 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 210/411] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:23:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:23:16.458715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90ad4b-f160-4a8f-8813-9813ab52c3b5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:16.458993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90ad4b-f160-4a8f-8813-9813ab52c3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:16.459263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90ad4b-f160-4a8f-8813-9813ab52c3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:16.478162 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-be90ad4b-f160-4a8f-8813-9813ab52c3b5 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:16.479634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-be90ad4b-f160-4a8f-8813-9813ab52c3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:16.546896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7be8c-774b-429c-bd24-56855916e52e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:16.550540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 211/412] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:16.551333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90ad4b-f160-4a8f-8813-9813ab52c3b5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:16.585320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101f504-2d9e-416e-97c3-5bb37c1e3c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:16.586459 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101f504-2d9e-416e-97c3-5bb37c1e3c80 None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:16.586930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101f504-2d9e-416e-97c3-5bb37c1e3c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:16.587491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101f504-2d9e-416e-97c3-5bb37c1e3c80 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6/groups/459bcbad31f44b8fad10cdd5de14856c/roles/be3ffa4db9954414868f116f97f3a929` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:16.589949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101f504-2d9e-416e-97c3-5bb37c1e3c80 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ab355ce83f94db6a7c315ba27ae52a6, group_id=459bcbad31f44b8fad10cdd5de14856c, role_id=be3ffa4db9954414868f116f97f3a929)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:16.591622 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101f504-2d9e-416e-97c3-5bb37c1e3c80 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:16.619246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 202/413] 10.222.0.62 () {58 vars in 1442 bytes} [Thu Sep 1 16:23:16 2022] PUT /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6/groups/459bcbad31f44b8fad10cdd5de14856c/roles/be3ffa4db9954414868f116f97f3a929 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:23:16.666423 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90ad4b-f160-4a8f-8813-9813ab52c3b5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:16.677995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:16.712928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2c6e9-1941-44ca-bce4-6d0bd1f4717b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:16.713782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2c6e9-1941-44ca-bce4-6d0bd1f4717b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:16.714063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2c6e9-1941-44ca-bce4-6d0bd1f4717b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:16.714331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2c6e9-1941-44ca-bce4-6d0bd1f4717b None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:16.715023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2c6e9-1941-44ca-bce4-6d0bd1f4717b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:16.715326 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2c6e9-1941-44ca-bce4-6d0bd1f4717b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:16.716838 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2c6e9-1941-44ca-bce4-6d0bd1f4717b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:16.726471 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 212/414] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:16.732293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101f504-2d9e-416e-97c3-5bb37c1e3c80 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:16.758999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:16.842736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2c6e9-1941-44ca-bce4-6d0bd1f4717b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:16.887485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101f504-2d9e-416e-97c3-5bb37c1e3c80 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:16.948064 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21db762-1d5e-4aa1-a843-7d4378d46fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:16.948638 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21db762-1d5e-4aa1-a843-7d4378d46fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:16.948911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21db762-1d5e-4aa1-a843-7d4378d46fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:16.949177 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21db762-1d5e-4aa1-a843-7d4378d46fe8 None admin] PATH_INFO: `/v3/groups/6670d385a6164f81a07228147028d866` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:16.950406 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21db762-1d5e-4aa1-a843-7d4378d46fe8 None admin] RBAC: Authorizing `identity:get_group(group_id=6670d385a6164f81a07228147028d866)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:16.951858 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21db762-1d5e-4aa1-a843-7d4378d46fe8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:16.958673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 203/415] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:16 2022] GET /identity/v3/groups/6670d385a6164f81a07228147028d866 => generated 229 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:16.959998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21db762-1d5e-4aa1-a843-7d4378d46fe8 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:17.038015 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2c6e9-1941-44ca-bce4-6d0bd1f4717b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:17.056359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:17.064923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21db762-1d5e-4aa1-a843-7d4378d46fe8 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:17.106284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72826688-cf27-427c-9a11-59d01650d70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:17.107000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b83f1-8483-468c-a0d4-f363cb10362c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:17.107353 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72826688-cf27-427c-9a11-59d01650d70b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:17.107704 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b83f1-8483-468c-a0d4-f363cb10362c None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:17.108086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72826688-cf27-427c-9a11-59d01650d70b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:17.108203 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b83f1-8483-468c-a0d4-f363cb10362c None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:17.108463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b83f1-8483-468c-a0d4-f363cb10362c None admin] PATH_INFO: `/v3/projects/026b4ed9c3cf48a48d86818cba23c13c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:17.108631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72826688-cf27-427c-9a11-59d01650d70b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:17.109853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b83f1-8483-468c-a0d4-f363cb10362c None admin] RBAC: Authorizing `identity:get_project(project_id=026b4ed9c3cf48a48d86818cba23c13c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:17.111475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b83f1-8483-468c-a0d4-f363cb10362c None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:17.118263 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 204/416] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:16 2022] GET /identity/v3/projects/026b4ed9c3cf48a48d86818cba23c13c => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:17.122493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b83f1-8483-468c-a0d4-f363cb10362c None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:17.135515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72826688-cf27-427c-9a11-59d01650d70b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-72826688-cf27-427c-9a11-59d01650d70b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:17.143420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:17.144344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:17.145119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:17.145119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:17.145119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:17.145119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:17.145119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:17.145119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:17.145119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:17.145119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:17.145119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:17.145119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:17.145119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:17.145119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:17.147256 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 213/417] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:17.152192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72826688-cf27-427c-9a11-59d01650d70b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:17.206049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:17.253856 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b83f1-8483-468c-a0d4-f363cb10362c None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:17.292140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0f39e-a719-4ecd-bada-b116613c4735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:17.292826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0f39e-a719-4ecd-bada-b116613c4735 None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:17.293151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0f39e-a719-4ecd-bada-b116613c4735 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:17.293435 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0f39e-a719-4ecd-bada-b116613c4735 None admin] PATH_INFO: `/v3/projects/026b4ed9c3cf48a48d86818cba23c13c/groups/6670d385a6164f81a07228147028d866/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:17.295400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0f39e-a719-4ecd-bada-b116613c4735 None admin] RBAC: Authorizing `identity:create_grant(project_id=026b4ed9c3cf48a48d86818cba23c13c, group_id=6670d385a6164f81a07228147028d866, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:17.297213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0f39e-a719-4ecd-bada-b116613c4735 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:17.320848 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 205/418] 10.222.0.62 () {58 vars in 1442 bytes} [Thu Sep 1 16:23:17 2022] PUT /identity/v3/projects/026b4ed9c3cf48a48d86818cba23c13c/groups/6670d385a6164f81a07228147028d866/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:23:17.388797 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72826688-cf27-427c-9a11-59d01650d70b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:17.452636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b4bd3-d942-4748-b8b9-63c24a58674a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:17.453280 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b4bd3-d942-4748-b8b9-63c24a58674a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:17.453556 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b4bd3-d942-4748-b8b9-63c24a58674a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:17.453827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b4bd3-d942-4748-b8b9-63c24a58674a None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:17.454520 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b4bd3-d942-4748-b8b9-63c24a58674a None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:17.454823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b4bd3-d942-4748-b8b9-63c24a58674a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:17.456336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b4bd3-d942-4748-b8b9-63c24a58674a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:17.472293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 214/419] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:23:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:17.478986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0f39e-a719-4ecd-bada-b116613c4735 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:17.498732 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:17.600265 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0f39e-a719-4ecd-bada-b116613c4735 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:17.635517 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94d239-04b7-4483-b2eb-cba6c69647a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:17.636225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94d239-04b7-4483-b2eb-cba6c69647a6 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:17.636499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94d239-04b7-4483-b2eb-cba6c69647a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:17.636769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94d239-04b7-4483-b2eb-cba6c69647a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:17.709101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da94d239-04b7-4483-b2eb-cba6c69647a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:17.710508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da94d239-04b7-4483-b2eb-cba6c69647a6 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:17.728521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:17.813273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 206/420] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:17.825052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b4bd3-d942-4748-b8b9-63c24a58674a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:17.973960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:17.984696 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b4bd3-d942-4748-b8b9-63c24a58674a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:18.022284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384fe43-4da2-4846-b262-5d0e50d3cd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:18.022929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384fe43-4da2-4846-b262-5d0e50d3cd8f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:18.023209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384fe43-4da2-4846-b262-5d0e50d3cd8f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:18.023478 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384fe43-4da2-4846-b262-5d0e50d3cd8f None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:18.024644 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384fe43-4da2-4846-b262-5d0e50d3cd8f None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:18.026272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384fe43-4da2-4846-b262-5d0e50d3cd8f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:18.028289 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 215/421] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:17 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:18.037248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94d239-04b7-4483-b2eb-cba6c69647a6 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:18.127618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6c1cc-b4cb-4bf8-a62a-e3666b9e413c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:18.128200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6c1cc-b4cb-4bf8-a62a-e3666b9e413c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:18.128474 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6c1cc-b4cb-4bf8-a62a-e3666b9e413c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:18.130144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 216/422] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:18.135310 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87176c67-02fb-475f-ae34-3ccf960e5eb7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:18.135590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87176c67-02fb-475f-ae34-3ccf960e5eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:18.135863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87176c67-02fb-475f-ae34-3ccf960e5eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:18.170900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-87176c67-02fb-475f-ae34-3ccf960e5eb7 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:18.172141 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-87176c67-02fb-475f-ae34-3ccf960e5eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:18.190980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94d239-04b7-4483-b2eb-cba6c69647a6 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:18.212428 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:18.234251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f27c5-40e1-431b-9b32-f787ef5662fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:18.234828 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f27c5-40e1-431b-9b32-f787ef5662fd None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:18.235172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f27c5-40e1-431b-9b32-f787ef5662fd None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:18.235450 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f27c5-40e1-431b-9b32-f787ef5662fd None admin] PATH_INFO: `/v3/groups/459bcbad31f44b8fad10cdd5de14856c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:18.236809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f27c5-40e1-431b-9b32-f787ef5662fd None admin] RBAC: Authorizing `identity:get_group(group_id=459bcbad31f44b8fad10cdd5de14856c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:18.238281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f27c5-40e1-431b-9b32-f787ef5662fd None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:18.244036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 207/423] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:18 2022] GET /identity/v3/groups/459bcbad31f44b8fad10cdd5de14856c => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:18.250657 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f27c5-40e1-431b-9b32-f787ef5662fd None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:18.307785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 217/424] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:23:18.406561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f27c5-40e1-431b-9b32-f787ef5662fd None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:18.455630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe1bc0-c60d-4d46-b2d7-d2ce31b994bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:18.456917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe1bc0-c60d-4d46-b2d7-d2ce31b994bf None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:18.457399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe1bc0-c60d-4d46-b2d7-d2ce31b994bf None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:18.457918 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe1bc0-c60d-4d46-b2d7-d2ce31b994bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:18.458903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe1bc0-c60d-4d46-b2d7-d2ce31b994bf None admin] RBAC: Adding query filter params (group.id=459bcbad31f44b8fad10cdd5de14856c, role.id=3100f6054f7d4d7b984fe19a4d8ba291, scope.project.id=ce0bf520fc554803bd196782f428d532) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:18.459412 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe1bc0-c60d-4d46-b2d7-d2ce31b994bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:18.461105 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe1bc0-c60d-4d46-b2d7-d2ce31b994bf None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:18.467766 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 208/425] 10.222.0.62 () {56 vars in 1376 bytes} [Thu Sep 1 16:23:18 2022] GET /identity/v3/role_assignments?group.id=459bcbad31f44b8fad10cdd5de14856c&scope.project.id=ce0bf520fc554803bd196782f428d532&role.id=3100f6054f7d4d7b984fe19a4d8ba291 => generated 611 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:18.609309 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87176c67-02fb-475f-ae34-3ccf960e5eb7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:18.690578 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316cd99-c645-40ec-9f09-d00e1cdbcc42 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:18.690920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316cd99-c645-40ec-9f09-d00e1cdbcc42 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:18.691119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316cd99-c645-40ec-9f09-d00e1cdbcc42 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:18.692833 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 209/426] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:18.699140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209eea3-5a74-423c-b205-095baf348e78 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:18.699491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209eea3-5a74-423c-b205-095baf348e78 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:18.699688 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209eea3-5a74-423c-b205-095baf348e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:18.717680 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:18.728579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87176c67-02fb-475f-ae34-3ccf960e5eb7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:18.729341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-f209eea3-5a74-423c-b205-095baf348e78 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:18.730581 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f209eea3-5a74-423c-b205-095baf348e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:18.781491 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3173cf0-cb7b-4f65-80b1-0eea4a0a3795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:18.782683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3173cf0-cb7b-4f65-80b1-0eea4a0a3795 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:18.783228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3173cf0-cb7b-4f65-80b1-0eea4a0a3795 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:18.783692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3173cf0-cb7b-4f65-80b1-0eea4a0a3795 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:18.786177 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:18.797082 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3173cf0-cb7b-4f65-80b1-0eea4a0a3795 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c3173cf0-cb7b-4f65-80b1-0eea4a0a3795 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:18.803314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:18.804286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:18.805697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:18.805697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:18.805697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:18.805697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:18.805697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:18.805697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:18.805697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:18.805697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:18.805697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:18.805697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:18.805697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:23:18.805697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:18.805697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 218/427] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:18.813068 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3173cf0-cb7b-4f65-80b1-0eea4a0a3795 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:18.839029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 210/428] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:23:18.927139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3173cf0-cb7b-4f65-80b1-0eea4a0a3795 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:18.949285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942ceb5-08a1-4ff6-a58c-1251412df004 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:18.949906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942ceb5-08a1-4ff6-a58c-1251412df004 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:18.950376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942ceb5-08a1-4ff6-a58c-1251412df004 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:18.952297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 211/429] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:23:18.958182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a92b20-fcae-410c-a021-ca5015628508 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:18.958708 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a92b20-fcae-410c-a021-ca5015628508 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:18.959166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a92b20-fcae-410c-a021-ca5015628508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:18.972086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485e288-0e94-4969-9274-9ebd0016a50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:18.972705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485e288-0e94-4969-9274-9ebd0016a50d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:18.972979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485e288-0e94-4969-9274-9ebd0016a50d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:18.973248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485e288-0e94-4969-9274-9ebd0016a50d None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:18.974348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485e288-0e94-4969-9274-9ebd0016a50d None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:18.974668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485e288-0e94-4969-9274-9ebd0016a50d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:18.976172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485e288-0e94-4969-9274-9ebd0016a50d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:18.983395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-35a92b20-fcae-410c-a021-ca5015628508 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:18.985273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-35a92b20-fcae-410c-a021-ca5015628508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:18.989177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 219/430] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:23:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:18.995451 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485e288-0e94-4969-9274-9ebd0016a50d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:19.019790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:19.077684 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 212/431] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:19.124558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485e288-0e94-4969-9274-9ebd0016a50d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:19.146467 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a92b20-fcae-410c-a021-ca5015628508 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:19.174590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835e46b-94f1-4fa2-b0f8-7999c5e64d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:19.175261 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835e46b-94f1-4fa2-b0f8-7999c5e64d83 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:19.175487 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835e46b-94f1-4fa2-b0f8-7999c5e64d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:19.176302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835e46b-94f1-4fa2-b0f8-7999c5e64d83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:19.260332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835e46b-94f1-4fa2-b0f8-7999c5e64d83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:19.261736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835e46b-94f1-4fa2-b0f8-7999c5e64d83 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:19.273748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a92b20-fcae-410c-a021-ca5015628508 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:19.275465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:19.277792 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:19.315053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816efb9-cbab-4eb2-835f-aa98a3bf7e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:19.315768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816efb9-cbab-4eb2-835f-aa98a3bf7e4e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:19.316097 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816efb9-cbab-4eb2-835f-aa98a3bf7e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:19.316365 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816efb9-cbab-4eb2-835f-aa98a3bf7e4e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:19.328030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816efb9-cbab-4eb2-835f-aa98a3bf7e4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-0816efb9-cbab-4eb2-835f-aa98a3bf7e4e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:19.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:19.334970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:19.336191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:19.336191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:19.336191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:19.336191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:19.336191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:19.336191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:19.336191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:19.336191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:19.336191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:19.336191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:19.336191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:23:19.336191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:19.336191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 213/432] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:23:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:19.342661 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816efb9-cbab-4eb2-835f-aa98a3bf7e4e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:19.379913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 220/433] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:23:19.385915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835e46b-94f1-4fa2-b0f8-7999c5e64d83 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:19.460860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816efb9-cbab-4eb2-835f-aa98a3bf7e4e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:19.496732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835e46b-94f1-4fa2-b0f8-7999c5e64d83 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:19.499008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e16ff2-d676-4ba9-862e-b3fef7147feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:19.500449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e16ff2-d676-4ba9-862e-b3fef7147feb None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:19.500732 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e16ff2-d676-4ba9-862e-b3fef7147feb None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:19.501004 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e16ff2-d676-4ba9-862e-b3fef7147feb None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:19.501671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e16ff2-d676-4ba9-862e-b3fef7147feb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:19.501975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e16ff2-d676-4ba9-862e-b3fef7147feb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:19.503385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e16ff2-d676-4ba9-862e-b3fef7147feb None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:19.510327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:19.519443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 214/434] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:19.520191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e16ff2-d676-4ba9-862e-b3fef7147feb None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:19.532737 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae00ec-5140-49ff-86ce-c4cf385812c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:19.533607 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae00ec-5140-49ff-86ce-c4cf385812c6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:19.534025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae00ec-5140-49ff-86ce-c4cf385812c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:19.534453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae00ec-5140-49ff-86ce-c4cf385812c6 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:19.535856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae00ec-5140-49ff-86ce-c4cf385812c6 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:19.537506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae00ec-5140-49ff-86ce-c4cf385812c6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:19.539231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 221/435] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:19 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:19.541711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae00ec-5140-49ff-86ce-c4cf385812c6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:19.549241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:19.632415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e16ff2-d676-4ba9-862e-b3fef7147feb None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:19.642988 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae00ec-5140-49ff-86ce-c4cf385812c6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:19.666127 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25945fe-1a52-49bb-82fd-c630f505aa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:19.666751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25945fe-1a52-49bb-82fd-c630f505aa64 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:19.667032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25945fe-1a52-49bb-82fd-c630f505aa64 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:19.667292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25945fe-1a52-49bb-82fd-c630f505aa64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:19.675789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c20cd-e295-478f-8bae-822dea9d959d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:19.676415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c20cd-e295-478f-8bae-822dea9d959d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:19.676687 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c20cd-e295-478f-8bae-822dea9d959d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:19.676955 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c20cd-e295-478f-8bae-822dea9d959d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:19.677254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25945fe-1a52-49bb-82fd-c630f505aa64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-d25945fe-1a52-49bb-82fd-c630f505aa64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:19.682756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:19.683787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:19.684791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:19.684791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:19.684791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:19.684791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:19.684791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:19.684791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:19.684791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:19.684791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:19.684791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:19.684791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:19.684791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:23:19.684791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:19.684791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 215/436] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:23:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:19.685569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25945fe-1a52-49bb-82fd-c630f505aa64 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:19.731334 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:19.753737 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c20cd-e295-478f-8bae-822dea9d959d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:19.755214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c20cd-e295-478f-8bae-822dea9d959d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:19.772272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:19.789609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25945fe-1a52-49bb-82fd-c630f505aa64 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:19.824562 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d465f-e56c-4ac8-8632-e5560c6b89ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:19.825186 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d465f-e56c-4ac8-8632-e5560c6b89ea None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:19.825461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d465f-e56c-4ac8-8632-e5560c6b89ea None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:19.825729 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d465f-e56c-4ac8-8632-e5560c6b89ea None admin] PATH_INFO: `/v3/groups/459bcbad31f44b8fad10cdd5de14856c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:19.826852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d465f-e56c-4ac8-8632-e5560c6b89ea None admin] RBAC: Authorizing `identity:get_group(group_id=459bcbad31f44b8fad10cdd5de14856c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:19.828343 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d465f-e56c-4ac8-8632-e5560c6b89ea None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:19.829925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 216/437] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:19 2022] GET /identity/v3/groups/459bcbad31f44b8fad10cdd5de14856c => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:19.833099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb2ddb-c7f1-409a-b7dd-f09089189587 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:19.833348 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb2ddb-c7f1-409a-b7dd-f09089189587 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:19.833628 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb2ddb-c7f1-409a-b7dd-f09089189587 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:19.835025 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 217/438] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:19.839621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb2ddb-c7f1-409a-b7dd-f09089189587 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:19.856980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 222/439] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:19.858051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c20cd-e295-478f-8bae-822dea9d959d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:19.943425 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:19.950769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb2ddb-c7f1-409a-b7dd-f09089189587 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:19.965323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c20cd-e295-478f-8bae-822dea9d959d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:19.985737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fba2e-5271-4d31-98e1-650387c1e880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:19.986357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fba2e-5271-4d31-98e1-650387c1e880 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:19.986634 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fba2e-5271-4d31-98e1-650387c1e880 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:19.986899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fba2e-5271-4d31-98e1-650387c1e880 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:19.987553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fba2e-5271-4d31-98e1-650387c1e880 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:19.987859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fba2e-5271-4d31-98e1-650387c1e880 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:19.989861 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fba2e-5271-4d31-98e1-650387c1e880 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:20.001579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 218/440] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:20.003016 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb98ab6-29fe-4baa-b6bc-b183a7effa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:20.003160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4cc21-23b8-45c1-9aff-3f54c5d114f6 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:20.003405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4cc21-23b8-45c1-9aff-3f54c5d114f6 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:20.003627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb98ab6-29fe-4baa-b6bc-b183a7effa19 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:20.003721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4cc21-23b8-45c1-9aff-3f54c5d114f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:20.003908 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb98ab6-29fe-4baa-b6bc-b183a7effa19 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:20.004198 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb98ab6-29fe-4baa-b6bc-b183a7effa19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:20.004917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb98ab6-29fe-4baa-b6bc-b183a7effa19 None admin] RBAC: Adding query filter params (group.id=459bcbad31f44b8fad10cdd5de14856c, role.id=3100f6054f7d4d7b984fe19a4d8ba291, scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:20.005217 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb98ab6-29fe-4baa-b6bc-b183a7effa19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:20.006646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb98ab6-29fe-4baa-b6bc-b183a7effa19 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:20.013506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 223/441] 10.222.0.62 () {56 vars in 1376 bytes} [Thu Sep 1 16:23:19 2022] GET /identity/v3/role_assignments?group.id=459bcbad31f44b8fad10cdd5de14856c&scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6&role.id=3100f6054f7d4d7b984fe19a4d8ba291 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:20.014301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb98ab6-29fe-4baa-b6bc-b183a7effa19 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:20.032213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-23a4cc21-23b8-45c1-9aff-3f54c5d114f6 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:20.033361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-23a4cc21-23b8-45c1-9aff-3f54c5d114f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:20.044924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:20.126337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 219/442] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:20.127214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4cc21-23b8-45c1-9aff-3f54c5d114f6 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:20.142318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb98ab6-29fe-4baa-b6bc-b183a7effa19 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:20.198260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df852f-93ee-4862-ba1b-422cbc61e346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:20.199061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df852f-93ee-4862-ba1b-422cbc61e346 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:20.199448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df852f-93ee-4862-ba1b-422cbc61e346 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:20.199717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df852f-93ee-4862-ba1b-422cbc61e346 None admin] PATH_INFO: `/v3/projects/ce0bf520fc554803bd196782f428d532` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:20.201320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df852f-93ee-4862-ba1b-422cbc61e346 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0bf520fc554803bd196782f428d532)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:20.203061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df852f-93ee-4862-ba1b-422cbc61e346 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:20.205094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 224/443] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:20 2022] GET /identity/v3/projects/ce0bf520fc554803bd196782f428d532 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:20.211578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df852f-93ee-4862-ba1b-422cbc61e346 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:20.232977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:20.256484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4cc21-23b8-45c1-9aff-3f54c5d114f6 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:20.297610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4235f-01a6-4008-a0f1-ca844dacfa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:20.298131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4235f-01a6-4008-a0f1-ca844dacfa0b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:20.298407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4235f-01a6-4008-a0f1-ca844dacfa0b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:20.298674 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4235f-01a6-4008-a0f1-ca844dacfa0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:20.313831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:20.332181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df852f-93ee-4862-ba1b-422cbc61e346 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:20.366988 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204ba51-d467-456e-9a2b-79f2a9b4385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:20.367739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204ba51-d467-456e-9a2b-79f2a9b4385f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:20.368107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204ba51-d467-456e-9a2b-79f2a9b4385f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:20.368438 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204ba51-d467-456e-9a2b-79f2a9b4385f None admin] PATH_INFO: `/v3/groups/459bcbad31f44b8fad10cdd5de14856c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:20.370296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204ba51-d467-456e-9a2b-79f2a9b4385f None admin] RBAC: Authorizing `identity:get_group(group_id=459bcbad31f44b8fad10cdd5de14856c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:20.371395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4235f-01a6-4008-a0f1-ca844dacfa0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:20.372878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204ba51-d467-456e-9a2b-79f2a9b4385f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:20.372976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4235f-01a6-4008-a0f1-ca844dacfa0b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:20.375549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 225/444] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:20 2022] GET /identity/v3/groups/459bcbad31f44b8fad10cdd5de14856c => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:20.381231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204ba51-d467-456e-9a2b-79f2a9b4385f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:20.461588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:20.475947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 220/445] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:20.476876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4235f-01a6-4008-a0f1-ca844dacfa0b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:20.484876 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204ba51-d467-456e-9a2b-79f2a9b4385f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:20.518431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b8f2d-e14d-4f88-9e15-2773ebf69e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:20.519055 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b8f2d-e14d-4f88-9e15-2773ebf69e51 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:20.519330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b8f2d-e14d-4f88-9e15-2773ebf69e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:20.519598 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b8f2d-e14d-4f88-9e15-2773ebf69e51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:20.520350 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b8f2d-e14d-4f88-9e15-2773ebf69e51 None admin] RBAC: Adding query filter params (group.id=459bcbad31f44b8fad10cdd5de14856c, role.id=be3ffa4db9954414868f116f97f3a929, scope.project.id=ce0bf520fc554803bd196782f428d532) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:20.520651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b8f2d-e14d-4f88-9e15-2773ebf69e51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:20.521616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:20.522083 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b8f2d-e14d-4f88-9e15-2773ebf69e51 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:20.528335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 226/446] 10.222.0.62 () {56 vars in 1376 bytes} [Thu Sep 1 16:23:20 2022] GET /identity/v3/role_assignments?group.id=459bcbad31f44b8fad10cdd5de14856c&scope.project.id=ce0bf520fc554803bd196782f428d532&role.id=be3ffa4db9954414868f116f97f3a929 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:20.529547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b8f2d-e14d-4f88-9e15-2773ebf69e51 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:20.592266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4235f-01a6-4008-a0f1-ca844dacfa0b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:20.630267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8a14f-2259-4e6f-b935-fffdf6a7bec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:20.630922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8a14f-2259-4e6f-b935-fffdf6a7bec5 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:20.631223 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8a14f-2259-4e6f-b935-fffdf6a7bec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:20.631917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8a14f-2259-4e6f-b935-fffdf6a7bec5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:20.644507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8a14f-2259-4e6f-b935-fffdf6a7bec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-b8e8a14f-2259-4e6f-b935-fffdf6a7bec5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:20.649654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:20.651497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:20.653027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:20.653027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:20.653027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:20.653027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:20.653027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:20.653027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:20.653027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:20.653027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:20.653027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:20.653027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:20.653027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:20.653027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:20.653027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 221/447] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:23:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:20.656884 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8a14f-2259-4e6f-b935-fffdf6a7bec5 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:20.659419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b8f2d-e14d-4f88-9e15-2773ebf69e51 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:20.700673 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f49dda-79f5-468b-8021-d97a8c552750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:20.701318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f49dda-79f5-468b-8021-d97a8c552750 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:20.701593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f49dda-79f5-468b-8021-d97a8c552750 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:20.701862 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f49dda-79f5-468b-8021-d97a8c552750 None admin] PATH_INFO: `/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:20.703119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f49dda-79f5-468b-8021-d97a8c552750 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab355ce83f94db6a7c315ba27ae52a6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:20.704928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f49dda-79f5-468b-8021-d97a8c552750 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:20.706379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 227/448] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:20 2022] GET /identity/v3/projects/7ab355ce83f94db6a7c315ba27ae52a6 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:20.712226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f49dda-79f5-468b-8021-d97a8c552750 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:20.729435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:20.786594 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8a14f-2259-4e6f-b935-fffdf6a7bec5 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:20.789242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:20.827831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c95dee-26e5-4564-8478-a99dda491818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:20.828476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c95dee-26e5-4564-8478-a99dda491818 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:20.828750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c95dee-26e5-4564-8478-a99dda491818 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:20.829018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c95dee-26e5-4564-8478-a99dda491818 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:20.829691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c95dee-26e5-4564-8478-a99dda491818 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:20.829993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c95dee-26e5-4564-8478-a99dda491818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:20.831388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c95dee-26e5-4564-8478-a99dda491818 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:20.840848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f49dda-79f5-468b-8021-d97a8c552750 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:20.847984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 222/449] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:20.850797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c95dee-26e5-4564-8478-a99dda491818 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:20.887468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0d2fd-7d5e-428d-b661-9c55689a17cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:20.889869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0d2fd-7d5e-428d-b661-9c55689a17cb None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:20.889869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0d2fd-7d5e-428d-b661-9c55689a17cb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:20.889869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0d2fd-7d5e-428d-b661-9c55689a17cb None admin] PATH_INFO: `/v3/groups/459bcbad31f44b8fad10cdd5de14856c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:20.890119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0d2fd-7d5e-428d-b661-9c55689a17cb None admin] RBAC: Authorizing `identity:get_group(group_id=459bcbad31f44b8fad10cdd5de14856c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:20.892211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0d2fd-7d5e-428d-b661-9c55689a17cb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:20.893070 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 228/450] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:20 2022] GET /identity/v3/groups/459bcbad31f44b8fad10cdd5de14856c => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:20.899028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0d2fd-7d5e-428d-b661-9c55689a17cb None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:20.969806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c95dee-26e5-4564-8478-a99dda491818 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:20.992351 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:21.008284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893a019-dcae-4334-8fd9-f404132fa878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:21.008898 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893a019-dcae-4334-8fd9-f404132fa878 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:21.009353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893a019-dcae-4334-8fd9-f404132fa878 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:21.009641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893a019-dcae-4334-8fd9-f404132fa878 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:21.015589 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0d2fd-7d5e-428d-b661-9c55689a17cb None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:21.047386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:21.056066 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd533b-5ab1-463f-9a85-8be1e78e5320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:21.056701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd533b-5ab1-463f-9a85-8be1e78e5320 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:21.056974 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd533b-5ab1-463f-9a85-8be1e78e5320 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:21.057238 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd533b-5ab1-463f-9a85-8be1e78e5320 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:21.057959 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd533b-5ab1-463f-9a85-8be1e78e5320 None admin] RBAC: Adding query filter params (group.id=459bcbad31f44b8fad10cdd5de14856c, role.id=be3ffa4db9954414868f116f97f3a929, scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:21.058262 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd533b-5ab1-463f-9a85-8be1e78e5320 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:21.059701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd533b-5ab1-463f-9a85-8be1e78e5320 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:21.065561 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 229/451] 10.222.0.62 () {56 vars in 1376 bytes} [Thu Sep 1 16:23:20 2022] GET /identity/v3/role_assignments?group.id=459bcbad31f44b8fad10cdd5de14856c&scope.project.id=7ab355ce83f94db6a7c315ba27ae52a6&role.id=be3ffa4db9954414868f116f97f3a929 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:21.090564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893a019-dcae-4334-8fd9-f404132fa878 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:21.092182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893a019-dcae-4334-8fd9-f404132fa878 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:21.203165 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 223/452] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:23:21.209471 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd533b-5ab1-463f-9a85-8be1e78e5320 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:21.316845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd533b-5ab1-463f-9a85-8be1e78e5320 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:21.350269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b5508-1b54-4390-af73-7ffa535c0aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:21.350905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b5508-1b54-4390-af73-7ffa535c0aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:21.351166 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b5508-1b54-4390-af73-7ffa535c0aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:21.351436 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b5508-1b54-4390-af73-7ffa535c0aa7 None admin] PATH_INFO: `/v3/projects/026b4ed9c3cf48a48d86818cba23c13c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:21.352574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b5508-1b54-4390-af73-7ffa535c0aa7 None admin] RBAC: Authorizing `identity:get_project(project_id=026b4ed9c3cf48a48d86818cba23c13c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:21.354294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b5508-1b54-4390-af73-7ffa535c0aa7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:21.355944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 230/453] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:23:21 2022] GET /identity/v3/projects/026b4ed9c3cf48a48d86818cba23c13c => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:21.361606 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893a019-dcae-4334-8fd9-f404132fa878 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:21.426179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:21.464829 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893a019-dcae-4334-8fd9-f404132fa878 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:21.499289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d84a56-0ebd-4a65-b806-92680af05be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:21.499928 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d84a56-0ebd-4a65-b806-92680af05be2 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:21.500201 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d84a56-0ebd-4a65-b806-92680af05be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:21.500460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d84a56-0ebd-4a65-b806-92680af05be2 None admin] PATH_INFO: `/v3/groups/6670d385a6164f81a07228147028d866` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:21.501560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d84a56-0ebd-4a65-b806-92680af05be2 None admin] RBAC: Authorizing `identity:get_group(group_id=6670d385a6164f81a07228147028d866)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:21.503022 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d84a56-0ebd-4a65-b806-92680af05be2 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:21.504611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 224/454] 10.222.0.62 () {56 vars in 1180 bytes} [Thu Sep 1 16:23:21 2022] GET /identity/v3/groups/6670d385a6164f81a07228147028d866 => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:21.511148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b5508-1b54-4390-af73-7ffa535c0aa7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:21.533797 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:21.635364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b5508-1b54-4390-af73-7ffa535c0aa7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:21.670439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67bc11-4858-414b-947c-abfbb1919d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:21.671037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67bc11-4858-414b-947c-abfbb1919d5b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:21.671316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67bc11-4858-414b-947c-abfbb1919d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:21.671589 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67bc11-4858-414b-947c-abfbb1919d5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:21.672334 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67bc11-4858-414b-947c-abfbb1919d5b None admin] RBAC: Adding query filter params (group.id=6670d385a6164f81a07228147028d866, role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=026b4ed9c3cf48a48d86818cba23c13c) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:21.672641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67bc11-4858-414b-947c-abfbb1919d5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:21.674068 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67bc11-4858-414b-947c-abfbb1919d5b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:21.680448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 231/455] 10.222.0.62 () {56 vars in 1376 bytes} [Thu Sep 1 16:23:21 2022] GET /identity/v3/role_assignments?group.id=6670d385a6164f81a07228147028d866&scope.project.id=026b4ed9c3cf48a48d86818cba23c13c&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:24.175137 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd7f2f-1228-46fc-a174-815ff48f6e26 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:24.176061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd7f2f-1228-46fc-a174-815ff48f6e26 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:24.176474 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd7f2f-1228-46fc-a174-815ff48f6e26 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:24.178300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 225/456] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:24.183284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958837ed-9ef8-4096-8a1d-266311b08f08 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:24.184130 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958837ed-9ef8-4096-8a1d-266311b08f08 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:24.184565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958837ed-9ef8-4096-8a1d-266311b08f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:24.203501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-958837ed-9ef8-4096-8a1d-266311b08f08 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:24.209233 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-958837ed-9ef8-4096-8a1d-266311b08f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:24.297406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 232/457] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:23:24.651057 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b11db8-18e1-4f50-9e1c-2a49e4d2410e None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:24.651430 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b11db8-18e1-4f50-9e1c-2a49e4d2410e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:24.651603 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b11db8-18e1-4f50-9e1c-2a49e4d2410e None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:24.653293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 226/458] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:24.669675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082cc57-3926-484a-b502-d0b734be6827 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:24.670238 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082cc57-3926-484a-b502-d0b734be6827 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:24.670728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082cc57-3926-484a-b502-d0b734be6827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:24.685225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b11db8-18e1-4f50-9e1c-2a49e4d2410e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:24.698959 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:24.707758 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2082cc57-3926-484a-b502-d0b734be6827 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:24.711465 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2082cc57-3926-484a-b502-d0b734be6827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:24.842553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 233/459] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:24.844441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af354306-dc20-4c8d-86ea-024dd747a749 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:24.844936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af354306-dc20-4c8d-86ea-024dd747a749 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:24.845414 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af354306-dc20-4c8d-86ea-024dd747a749 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:24.850998 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 234/460] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:24.852407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e0e9e-607b-4c9a-939c-c1e0c81e8a4c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:24.852879 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e0e9e-607b-4c9a-939c-c1e0c81e8a4c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:24.853527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e0e9e-607b-4c9a-939c-c1e0c81e8a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:24.861237 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b11db8-18e1-4f50-9e1c-2a49e4d2410e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:24.874657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a50e0e9e-607b-4c9a-939c-c1e0c81e8a4c None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:24.876111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a50e0e9e-607b-4c9a-939c-c1e0c81e8a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:24.897422 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfeaac-89e7-4bbb-98ee-f332ff4d2d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:24.898294 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfeaac-89e7-4bbb-98ee-f332ff4d2d9b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:24.898729 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfeaac-89e7-4bbb-98ee-f332ff4d2d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:24.899206 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfeaac-89e7-4bbb-98ee-f332ff4d2d9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:24.901657 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:24.910974 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfeaac-89e7-4bbb-98ee-f332ff4d2d9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:24.912713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfeaac-89e7-4bbb-98ee-f332ff4d2d9b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-58dfeaac-89e7-4bbb-98ee-f332ff4d2d9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:24.929633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:24.930681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:24.931589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:24.931589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:24.931589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:24.933930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 227/461] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:24.945807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfeaac-89e7-4bbb-98ee-f332ff4d2d9b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:24.964559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 235/462] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:25.053782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfeaac-89e7-4bbb-98ee-f332ff4d2d9b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:25.085360 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab796a7f-5e30-4f36-b619-8cdb1c6869a9 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:25.085653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab796a7f-5e30-4f36-b619-8cdb1c6869a9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:25.085898 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab796a7f-5e30-4f36-b619-8cdb1c6869a9 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:25.087792 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 236/463] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:25.092419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783df657-abf6-4be1-a850-3f8e0db2a83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:25.093031 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783df657-abf6-4be1-a850-3f8e0db2a83e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:25.093301 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783df657-abf6-4be1-a850-3f8e0db2a83e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:25.093567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783df657-abf6-4be1-a850-3f8e0db2a83e None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:25.094206 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783df657-abf6-4be1-a850-3f8e0db2a83e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:25.094503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783df657-abf6-4be1-a850-3f8e0db2a83e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:25.096031 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783df657-abf6-4be1-a850-3f8e0db2a83e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:25.101178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e482a-5cc2-40d8-a37d-a02af2ff6d69 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:25.101463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e482a-5cc2-40d8-a37d-a02af2ff6d69 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:25.101744 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e482a-5cc2-40d8-a37d-a02af2ff6d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:25.108805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 228/464] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:25.114553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783df657-abf6-4be1-a850-3f8e0db2a83e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:25.140930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-960e482a-5cc2-40d8-a37d-a02af2ff6d69 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:25.142141 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-960e482a-5cc2-40d8-a37d-a02af2ff6d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:25.168387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:25.212132 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:25.222179 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 237/465] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:23:25.223336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e482a-5cc2-40d8-a37d-a02af2ff6d69 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:25.227753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783df657-abf6-4be1-a850-3f8e0db2a83e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:25.273688 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690d053-99a8-41d5-8b9b-925476686ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:25.274448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690d053-99a8-41d5-8b9b-925476686ac2 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:25.274841 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690d053-99a8-41d5-8b9b-925476686ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:25.275369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690d053-99a8-41d5-8b9b-925476686ac2 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:25.276153 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690d053-99a8-41d5-8b9b-925476686ac2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:25.278386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690d053-99a8-41d5-8b9b-925476686ac2 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:25.316774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 229/466] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:23:25 2022] POST /identity/v3/users => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:23:25.354481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e482a-5cc2-40d8-a37d-a02af2ff6d69 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:25.404039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fab61-1b87-4029-b9ff-ed34c2b89242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:25.405088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fab61-1b87-4029-b9ff-ed34c2b89242 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:25.405707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fab61-1b87-4029-b9ff-ed34c2b89242 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:25.406141 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fab61-1b87-4029-b9ff-ed34c2b89242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:25.421987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526fab61-1b87-4029-b9ff-ed34c2b89242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:25.423689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526fab61-1b87-4029-b9ff-ed34c2b89242 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-526fab61-1b87-4029-b9ff-ed34c2b89242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:25.441582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:25.442729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:25.443765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:25.443765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:25.443765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:25.445569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 238/467] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:25.451713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690d053-99a8-41d5-8b9b-925476686ac2 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:25.522134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fab61-1b87-4029-b9ff-ed34c2b89242 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:25.572886 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690d053-99a8-41d5-8b9b-925476686ac2 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:25.575046 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:25.587848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:25.618191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78abb6b-250b-4f56-ab9d-ca44e687d122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:25.619227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78abb6b-250b-4f56-ab9d-ca44e687d122 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:25.619648 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78abb6b-250b-4f56-ab9d-ca44e687d122 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:25.619949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78abb6b-250b-4f56-ab9d-ca44e687d122 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:25.620604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78abb6b-250b-4f56-ab9d-ca44e687d122 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:25.620890 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78abb6b-250b-4f56-ab9d-ca44e687d122 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:25.622326 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78abb6b-250b-4f56-ab9d-ca44e687d122 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:25.636911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 230/468] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:25.637664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78abb6b-250b-4f56-ab9d-ca44e687d122 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:25.656148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fab61-1b87-4029-b9ff-ed34c2b89242 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:25.704781 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abd634-3ba8-444b-a4d8-e866279d0950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:25.705709 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abd634-3ba8-444b-a4d8-e866279d0950 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:25.705983 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abd634-3ba8-444b-a4d8-e866279d0950 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:25.706678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abd634-3ba8-444b-a4d8-e866279d0950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:25.722928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abd634-3ba8-444b-a4d8-e866279d0950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:25.724435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abd634-3ba8-444b-a4d8-e866279d0950 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-32abd634-3ba8-444b-a4d8-e866279d0950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:25.744206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:25.745396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:25.746359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:25.746359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:25.746359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:25.746359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 239/469] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:23:25.746710 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abd634-3ba8-444b-a4d8-e866279d0950 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:25.762591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78abb6b-250b-4f56-ab9d-ca44e687d122 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:25.797925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280499fb-953e-4cd9-a082-4c71f36c7b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:25.798782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280499fb-953e-4cd9-a082-4c71f36c7b6c None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:25.799147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280499fb-953e-4cd9-a082-4c71f36c7b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:25.799503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280499fb-953e-4cd9-a082-4c71f36c7b6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:25.812593 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280499fb-953e-4cd9-a082-4c71f36c7b6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:25.814188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280499fb-953e-4cd9-a082-4c71f36c7b6c None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-280499fb-953e-4cd9-a082-4c71f36c7b6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:25.830468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:25.831417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:25.832644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:25.832644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:25.832644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:25.832644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 231/470] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:25.832852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280499fb-953e-4cd9-a082-4c71f36c7b6c None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:25.851747 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:25.867362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:25.871399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abd634-3ba8-444b-a4d8-e866279d0950 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:25.909673 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce115a15-30c6-46d8-825c-e58533301131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:25.910275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce115a15-30c6-46d8-825c-e58533301131 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:25.910555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce115a15-30c6-46d8-825c-e58533301131 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:25.910830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce115a15-30c6-46d8-825c-e58533301131 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:25.911480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce115a15-30c6-46d8-825c-e58533301131 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:25.912955 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce115a15-30c6-46d8-825c-e58533301131 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:25.936420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 240/471] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:23:25 2022] POST /identity/v3/users => generated 249 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:23:25.937668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fc396-d2e1-4310-a412-ac7f9da5a48e None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:25.937950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fc396-d2e1-4310-a412-ac7f9da5a48e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:25.938221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fc396-d2e1-4310-a412-ac7f9da5a48e None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:25.939623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 241/472] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:25.944608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fc396-d2e1-4310-a412-ac7f9da5a48e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:25.955867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280499fb-953e-4cd9-a082-4c71f36c7b6c None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:26.001183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0087f-9da4-4713-b297-abbb5ed8cb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:26.002414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0087f-9da4-4713-b297-abbb5ed8cb6a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:26.002414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0087f-9da4-4713-b297-abbb5ed8cb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:26.002414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0087f-9da4-4713-b297-abbb5ed8cb6a None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:26.003011 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0087f-9da4-4713-b297-abbb5ed8cb6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:26.003314 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0087f-9da4-4713-b297-abbb5ed8cb6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:26.004858 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0087f-9da4-4713-b297-abbb5ed8cb6a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:26.018977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 232/473] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:26.020208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7e6e3-eab0-4e5f-84f7-2f8ba7013862 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:26.020544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7e6e3-eab0-4e5f-84f7-2f8ba7013862 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:26.020758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7e6e3-eab0-4e5f-84f7-2f8ba7013862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:26.040377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-8bd7e6e3-eab0-4e5f-84f7-2f8ba7013862 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:26.041714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8bd7e6e3-eab0-4e5f-84f7-2f8ba7013862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:26.097932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fc396-d2e1-4310-a412-ac7f9da5a48e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:26.128347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 233/474] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:26.129499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7e6e3-eab0-4e5f-84f7-2f8ba7013862 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:26.132875 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:26.141419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a564128-59a2-45b9-8ca6-85db4203944e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:26.142416 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a564128-59a2-45b9-8ca6-85db4203944e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:26.142965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a564128-59a2-45b9-8ca6-85db4203944e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:26.143445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a564128-59a2-45b9-8ca6-85db4203944e None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:26.144409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a564128-59a2-45b9-8ca6-85db4203944e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:26.144968 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a564128-59a2-45b9-8ca6-85db4203944e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:26.146823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a564128-59a2-45b9-8ca6-85db4203944e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:26.160875 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 242/475] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:26.169813 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a564128-59a2-45b9-8ca6-85db4203944e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:26.194591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:26.247846 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7e6e3-eab0-4e5f-84f7-2f8ba7013862 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:26.290277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a06628-e19c-4c3b-af73-b451753046d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:26.290941 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a06628-e19c-4c3b-af73-b451753046d6 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:26.291289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a06628-e19c-4c3b-af73-b451753046d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:26.291624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a06628-e19c-4c3b-af73-b451753046d6 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:26.292394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a06628-e19c-4c3b-af73-b451753046d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:26.293997 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a06628-e19c-4c3b-af73-b451753046d6 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:26.303755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a564128-59a2-45b9-8ca6-85db4203944e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:26.323326 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 234/476] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:23:26 2022] POST /identity/v3/users => generated 248 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:23:26.351931 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46f050-ba8c-4107-9e17-88eccc0824ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:26.352819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f050-ba8c-4107-9e17-88eccc0824ac None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:26.353457 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f050-ba8c-4107-9e17-88eccc0824ac None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:26.353886 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f050-ba8c-4107-9e17-88eccc0824ac None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:26.354958 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46f050-ba8c-4107-9e17-88eccc0824ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:26.356745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46f050-ba8c-4107-9e17-88eccc0824ac None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:26.385522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 243/477] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:23:26 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:23:26.478641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a06628-e19c-4c3b-af73-b451753046d6 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:26.556214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:26.606358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a06628-e19c-4c3b-af73-b451753046d6 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:26.643209 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211704b-a02c-40b3-8549-9411e9f75f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:26.643801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211704b-a02c-40b3-8549-9411e9f75f28 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:26.644102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211704b-a02c-40b3-8549-9411e9f75f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:26.644425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211704b-a02c-40b3-8549-9411e9f75f28 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:26.645147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211704b-a02c-40b3-8549-9411e9f75f28 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:26.646532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211704b-a02c-40b3-8549-9411e9f75f28 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:26.656301 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.sql.core [None req-e211704b-a02c-40b3-8549-9411e9f75f28 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 16:23:26.656301 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 16:23:26.656301 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [parameters: {'id': '2fa37146d6264460a05a67a4cb5fbc86', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 16:23:26.656301 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81523) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-e211704b-a02c-40b3-8549-9411e9f75f28 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application result.read() Sep 01 16:23:26.673635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:23:26.674575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application yield resource Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise self.value Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application session.commit() Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 16:23:26.675446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.session.flush() Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application compat.raise_( Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 16:23:26.676937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:23:26.678007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application result.read() Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:23:26.679042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [parameters: {'id': '2fa37146d6264460a05a67a4cb5fbc86', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:26.680489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 16:23:26.682493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:26.685116 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 235/478] 10.222.0.62 () {60 vars in 1131 bytes} [Thu Sep 1 16:23:26 2022] POST /identity/v3/roles => generated 144 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:26.685266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46f050-ba8c-4107-9e17-88eccc0824ac None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:26.775942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:26.788537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46f050-ba8c-4107-9e17-88eccc0824ac None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:26.824366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11257ea5-1130-4423-9ba3-8c7ecbb54018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:26.825006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11257ea5-1130-4423-9ba3-8c7ecbb54018 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:26.825279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11257ea5-1130-4423-9ba3-8c7ecbb54018 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:26.825543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11257ea5-1130-4423-9ba3-8c7ecbb54018 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:26.836278 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11257ea5-1130-4423-9ba3-8c7ecbb54018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-11257ea5-1130-4423-9ba3-8c7ecbb54018 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:26.841215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:26.843230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:26.844405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:26.844405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:26.844405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:26.844405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:26.844405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:26.844405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:26.844405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:26.844405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:26.844405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:26.844405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:26.844405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:23:26.844405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:26.844405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 244/479] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:23:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:26.849192 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211704b-a02c-40b3-8549-9411e9f75f28 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:27.010359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211704b-a02c-40b3-8549-9411e9f75f28 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:27.047615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:27.060442 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e99ad1-9803-4ad9-bf53-3184f44b44f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:27.061072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e99ad1-9803-4ad9-bf53-3184f44b44f4 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:27.061351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e99ad1-9803-4ad9-bf53-3184f44b44f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:27.061612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e99ad1-9803-4ad9-bf53-3184f44b44f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:27.062286 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e99ad1-9803-4ad9-bf53-3184f44b44f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:27.062584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e99ad1-9803-4ad9-bf53-3184f44b44f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:27.063999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e99ad1-9803-4ad9-bf53-3184f44b44f4 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:27.076943 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 236/480] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:27.576884 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08740e7a-c681-40dd-a0d5-7599005a7371 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:27.577414 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08740e7a-c681-40dd-a0d5-7599005a7371 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:27.577414 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08740e7a-c681-40dd-a0d5-7599005a7371 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:27.579111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 245/481] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:27.585043 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cf4ad-5bb6-423c-8ad1-efe14caee41f None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:27.585443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cf4ad-5bb6-423c-8ad1-efe14caee41f None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:27.585629 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cf4ad-5bb6-423c-8ad1-efe14caee41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:27.606515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-7a3cf4ad-5bb6-423c-8ad1-efe14caee41f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:27.608040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7a3cf4ad-5bb6-423c-8ad1-efe14caee41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:27.690266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 237/482] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:28.132173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08740e7a-c681-40dd-a0d5-7599005a7371 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:28.236571 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08740e7a-c681-40dd-a0d5-7599005a7371 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:28.268137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:28.274390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38bd36-e18d-4de9-9320-170f79dba041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:28.275354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38bd36-e18d-4de9-9320-170f79dba041 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:28.275640 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38bd36-e18d-4de9-9320-170f79dba041 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:28.275924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38bd36-e18d-4de9-9320-170f79dba041 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:28.286405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38bd36-e18d-4de9-9320-170f79dba041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-df38bd36-e18d-4de9-9320-170f79dba041 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:28.291304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:28.292282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:28.293081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:28.293081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:28.293081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:28.293081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:28.293081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:28.293081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:28.293081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:28.293081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:28.293081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:28.293081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:28.293081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:28.293081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:28.293081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 246/483] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:28.298769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cf4ad-5bb6-423c-8ad1-efe14caee41f None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:28.368339 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b3e86-3eb0-4143-aebd-b9e4c0660771 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:28.368717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b3e86-3eb0-4143-aebd-b9e4c0660771 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:28.368912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b3e86-3eb0-4143-aebd-b9e4c0660771 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:28.372291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 247/484] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:23:28.376470 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e423eaa-5d5e-40b3-96a1-b73bace00974 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:28.376755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e423eaa-5d5e-40b3-96a1-b73bace00974 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:28.377099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e423eaa-5d5e-40b3-96a1-b73bace00974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:28.397098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9e423eaa-5d5e-40b3-96a1-b73bace00974 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:28.398383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9e423eaa-5d5e-40b3-96a1-b73bace00974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:28.404849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cf4ad-5bb6-423c-8ad1-efe14caee41f None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:28.428390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:28.451765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eed89c-cc50-402b-950f-be472286e66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:28.452449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eed89c-cc50-402b-950f-be472286e66b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:28.452767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eed89c-cc50-402b-950f-be472286e66b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:28.453038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eed89c-cc50-402b-950f-be472286e66b None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:28.454184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eed89c-cc50-402b-950f-be472286e66b None admin] RBAC: Adding query filter params (name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:28.454488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eed89c-cc50-402b-950f-be472286e66b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:28.456146 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eed89c-cc50-402b-950f-be472286e66b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:28.467842 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 238/485] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:28.473213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eed89c-cc50-402b-950f-be472286e66b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:28.488038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 248/486] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:23:28.573846 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eed89c-cc50-402b-950f-be472286e66b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:28.608194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bd46d-ba24-4252-90c5-d6cbc40aa584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:28.608812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bd46d-ba24-4252-90c5-d6cbc40aa584 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:28.609128 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bd46d-ba24-4252-90c5-d6cbc40aa584 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:28.609338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bd46d-ba24-4252-90c5-d6cbc40aa584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:28.620357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bd46d-ba24-4252-90c5-d6cbc40aa584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:28.621799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bd46d-ba24-4252-90c5-d6cbc40aa584 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-4a0bd46d-ba24-4252-90c5-d6cbc40aa584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:28.636810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:28.637774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:28.639077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:28.639077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:28.639077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:28.639077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 239/487] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:28.643994 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e423eaa-5d5e-40b3-96a1-b73bace00974 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:28.777953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e423eaa-5d5e-40b3-96a1-b73bace00974 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:28.794715 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bd46d-ba24-4252-90c5-d6cbc40aa584 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:28.814641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:28.840765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2334c-cab7-4816-9bf6-d77d32ee8c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:28.842037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2334c-cab7-4816-9bf6-d77d32ee8c40 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:28.843218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2334c-cab7-4816-9bf6-d77d32ee8c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:28.843911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2334c-cab7-4816-9bf6-d77d32ee8c40 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:28.845832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2334c-cab7-4816-9bf6-d77d32ee8c40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:28.846521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2334c-cab7-4816-9bf6-d77d32ee8c40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:28.848725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2334c-cab7-4816-9bf6-d77d32ee8c40 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:28.854611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:28.875101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 249/488] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:28.880903 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2334c-cab7-4816-9bf6-d77d32ee8c40 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:28.983402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bd46d-ba24-4252-90c5-d6cbc40aa584 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:29.020849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0848f9-aa31-427b-8460-acba10be9b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:29.021435 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0848f9-aa31-427b-8460-acba10be9b48 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:29.021723 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0848f9-aa31-427b-8460-acba10be9b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:29.021994 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0848f9-aa31-427b-8460-acba10be9b48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:29.033767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0848f9-aa31-427b-8460-acba10be9b48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:29.038704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2334c-cab7-4816-9bf6-d77d32ee8c40 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-8d0848f9-aa31-427b-8460-acba10be9b48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:29.039074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:29.039976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:29.041008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:29.041008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:29.041008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:29.041008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:29.041008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:29.041008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:29.041008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:29.041008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:29.041008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:29.041008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:29.041008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:29.041008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:29.041008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 240/489] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:29.042390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112ea77-b5f8-40ef-98b6-2dc84ba8577d None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:29.042390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112ea77-b5f8-40ef-98b6-2dc84ba8577d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:29.042390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112ea77-b5f8-40ef-98b6-2dc84ba8577d None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:29.046453 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 241/490] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:29.046453 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f545b10-fa7b-4c02-97bb-af8b2d3c2838 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:29.046453 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f545b10-fa7b-4c02-97bb-af8b2d3c2838 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:29.046453 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f545b10-fa7b-4c02-97bb-af8b2d3c2838 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:29.046453 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 242/491] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:29.046939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f545b10-fa7b-4c02-97bb-af8b2d3c2838 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:29.083159 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf46b8a-6e52-436d-9c41-da7224a33c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:29.084369 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf46b8a-6e52-436d-9c41-da7224a33c6e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:29.084652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf46b8a-6e52-436d-9c41-da7224a33c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:29.084920 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf46b8a-6e52-436d-9c41-da7224a33c6e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:29.092867 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf46b8a-6e52-436d-9c41-da7224a33c6e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:29.094331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf46b8a-6e52-436d-9c41-da7224a33c6e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-edf46b8a-6e52-436d-9c41-da7224a33c6e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:29.107626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:29.108618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:29.109608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:29.109608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:29.109608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:29.109608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:29.109608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:29.109608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:29.109608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:29.109608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:29.109608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:23:29.109608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:29.112143 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 250/492] 10.222.0.62 () {56 vars in 1093 bytes} [Thu Sep 1 16:23:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:29.114629 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2fbf7-a7e5-4140-be07-5c1afbee5c50 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:29.115085 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2fbf7-a7e5-4140-be07-5c1afbee5c50 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:29.115378 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2fbf7-a7e5-4140-be07-5c1afbee5c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:29.136783 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-87a2fbf7-a7e5-4140-be07-5c1afbee5c50 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:29.138291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-87a2fbf7-a7e5-4140-be07-5c1afbee5c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:29.148053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:29.160128 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f545b10-fa7b-4c02-97bb-af8b2d3c2838 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:29.192562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:29.194766 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e2815-a310-4301-bb41-6d2a7407a21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:29.195431 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e2815-a310-4301-bb41-6d2a7407a21a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:29.195743 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e2815-a310-4301-bb41-6d2a7407a21a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:29.196072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e2815-a310-4301-bb41-6d2a7407a21a None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:29.196893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e2815-a310-4301-bb41-6d2a7407a21a None admin] RBAC: Adding query filter params (name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:29.197188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e2815-a310-4301-bb41-6d2a7407a21a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:29.198667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e2815-a310-4301-bb41-6d2a7407a21a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:29.212896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 243/493] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:29.212896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b8528-0bbb-4ebc-83ba-a46c9744aa49 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:29.212896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b8528-0bbb-4ebc-83ba-a46c9744aa49 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:29.212896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b8528-0bbb-4ebc-83ba-a46c9744aa49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:29.228506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-535b8528-0bbb-4ebc-83ba-a46c9744aa49 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:29.229673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-535b8528-0bbb-4ebc-83ba-a46c9744aa49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:29.247914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 251/494] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:29.247914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2fbf7-a7e5-4140-be07-5c1afbee5c50 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:29.329322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 244/495] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:29.330337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b8528-0bbb-4ebc-83ba-a46c9744aa49 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:29.388454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2fbf7-a7e5-4140-be07-5c1afbee5c50 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:29.461476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:29.472603 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca0b4d-e1e7-4a21-9803-12b6e3a4f339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:29.473228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca0b4d-e1e7-4a21-9803-12b6e3a4f339 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:29.473509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca0b4d-e1e7-4a21-9803-12b6e3a4f339 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:29.473836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca0b4d-e1e7-4a21-9803-12b6e3a4f339 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:29.474513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca0b4d-e1e7-4a21-9803-12b6e3a4f339 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:29.474827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca0b4d-e1e7-4a21-9803-12b6e3a4f339 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:29.476282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca0b4d-e1e7-4a21-9803-12b6e3a4f339 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:29.484831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b8528-0bbb-4ebc-83ba-a46c9744aa49 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:29.545500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 252/496] 10.222.0.62 () {56 vars in 1133 bytes} [Thu Sep 1 16:23:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:29.551726 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278835e6-a609-494c-b77b-2d5eead91833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:29.552476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278835e6-a609-494c-b77b-2d5eead91833 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:29.552801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278835e6-a609-494c-b77b-2d5eead91833 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:29.552919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca0b4d-e1e7-4a21-9803-12b6e3a4f339 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:29.553072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278835e6-a609-494c-b77b-2d5eead91833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:29.563775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278835e6-a609-494c-b77b-2d5eead91833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:29.565293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278835e6-a609-494c-b77b-2d5eead91833 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-278835e6-a609-494c-b77b-2d5eead91833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:29.580420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:29.581327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:29.582493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:29.582493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:29.582493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:29.586658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 245/497] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:29.587569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278835e6-a609-494c-b77b-2d5eead91833 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:29.591203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:29.668297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca0b4d-e1e7-4a21-9803-12b6e3a4f339 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:29.704424 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909fa93-d6f5-4699-a348-0a5f5375a2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:29.705135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909fa93-d6f5-4699-a348-0a5f5375a2db None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:29.705444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909fa93-d6f5-4699-a348-0a5f5375a2db None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:29.705771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909fa93-d6f5-4699-a348-0a5f5375a2db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:29.707252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278835e6-a609-494c-b77b-2d5eead91833 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:29.744356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6c72a-80a7-47ef-af64-41fe1c022a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:29.744995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6c72a-80a7-47ef-af64-41fe1c022a44 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:29.745275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6c72a-80a7-47ef-af64-41fe1c022a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:29.745544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6c72a-80a7-47ef-af64-41fe1c022a44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:29.747493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:29.764586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6c72a-80a7-47ef-af64-41fe1c022a44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-c7c6c72a-80a7-47ef-af64-41fe1c022a44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:29.770053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:29.770977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:29.772488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:29.772488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:29.772488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:29.772488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:29.772488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:29.772488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:29.772488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:29.772488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:29.772488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:29.772488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:29.772488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:29.772488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:29.772488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 246/498] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:29.772488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6c72a-80a7-47ef-af64-41fe1c022a44 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:29.782319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909fa93-d6f5-4699-a348-0a5f5375a2db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:29.783756 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909fa93-d6f5-4699-a348-0a5f5375a2db None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:29.813429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:29.877099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6c72a-80a7-47ef-af64-41fe1c022a44 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:29.883027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 253/499] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:29.884953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909fa93-d6f5-4699-a348-0a5f5375a2db None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:29.911868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6566f-a1ea-4205-a0a5-7b590690f033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:29.912782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6566f-a1ea-4205-a0a5-7b590690f033 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:29.913065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6566f-a1ea-4205-a0a5-7b590690f033 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:29.913318 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6566f-a1ea-4205-a0a5-7b590690f033 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:29.913964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6566f-a1ea-4205-a0a5-7b590690f033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:29.914271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6566f-a1ea-4205-a0a5-7b590690f033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:29.915719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6566f-a1ea-4205-a0a5-7b590690f033 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:29.928951 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 247/500] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:29.930146 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd5709-4467-4011-a2bf-41eb118b11d0 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:29.930373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd5709-4467-4011-a2bf-41eb118b11d0 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:29.930674 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd5709-4467-4011-a2bf-41eb118b11d0 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:29.932089 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 248/501] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:29.932795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd5709-4467-4011-a2bf-41eb118b11d0 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:29.989936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:29.990783 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909fa93-d6f5-4699-a348-0a5f5375a2db None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:30.020010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:30.025951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f8ba2-1747-404a-a206-1aa441251bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:30.026582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f8ba2-1747-404a-a206-1aa441251bfb None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:30.026857 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f8ba2-1747-404a-a206-1aa441251bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:30.027124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f8ba2-1747-404a-a206-1aa441251bfb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:30.034337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd5709-4467-4011-a2bf-41eb118b11d0 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:30.037714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f8ba2-1747-404a-a206-1aa441251bfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-101f8ba2-1747-404a-a206-1aa441251bfb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:30.042651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:30.043646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:30.044506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:30.044506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:30.044506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:30.044506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:30.044506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:30.044506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:30.044506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:30.044506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:30.044506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:30.044506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:30.044506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:30.044506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:30.044506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 254/502] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:30.045153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f8ba2-1747-404a-a206-1aa441251bfb None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:30.072539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237203ee-99ad-4945-b869-c1c904a8a026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:30.073151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237203ee-99ad-4945-b869-c1c904a8a026 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:30.073425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237203ee-99ad-4945-b869-c1c904a8a026 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:30.073689 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237203ee-99ad-4945-b869-c1c904a8a026 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:30.074345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237203ee-99ad-4945-b869-c1c904a8a026 None admin] RBAC: Adding query filter params (name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:30.074644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237203ee-99ad-4945-b869-c1c904a8a026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:30.076052 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237203ee-99ad-4945-b869-c1c904a8a026 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:30.084576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 249/503] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:30.085322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237203ee-99ad-4945-b869-c1c904a8a026 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:30.149894 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f8ba2-1747-404a-a206-1aa441251bfb None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:30.182509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425de3f4-2e07-416e-8871-6dbd26241e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:30.183128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425de3f4-2e07-416e-8871-6dbd26241e5e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:30.183398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425de3f4-2e07-416e-8871-6dbd26241e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:30.183661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425de3f4-2e07-416e-8871-6dbd26241e5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:30.190514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237203ee-99ad-4945-b869-c1c904a8a026 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:30.193523 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425de3f4-2e07-416e-8871-6dbd26241e5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:30.194966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425de3f4-2e07-416e-8871-6dbd26241e5e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:30.207210 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-425de3f4-2e07-416e-8871-6dbd26241e5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:30.209626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:30.210507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:30.211327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:30.211327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:30.211327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:30.211327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 255/504] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:30.211848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b72cc1-aef7-4129-95b8-9cbc61291e70 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:30.212126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b72cc1-aef7-4129-95b8-9cbc61291e70 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:30.212944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b72cc1-aef7-4129-95b8-9cbc61291e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:30.227354 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b83a2-f7fd-4dc2-99b2-cf0d55ed4990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:30.228095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b83a2-f7fd-4dc2-99b2-cf0d55ed4990 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:30.228457 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b83a2-f7fd-4dc2-99b2-cf0d55ed4990 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:30.228770 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b83a2-f7fd-4dc2-99b2-cf0d55ed4990 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:30.233962 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-48b72cc1-aef7-4129-95b8-9cbc61291e70 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:30.235153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-48b72cc1-aef7-4129-95b8-9cbc61291e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:30.236809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b83a2-f7fd-4dc2-99b2-cf0d55ed4990 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:30.238255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b83a2-f7fd-4dc2-99b2-cf0d55ed4990 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-b62b83a2-f7fd-4dc2-99b2-cf0d55ed4990 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:30.250702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:30.252291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:30.253239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:30.253239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:30.253239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:30.253239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:30.253239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:30.253239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:30.253239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:30.253239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:30.253239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:23:30.253239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:30.253239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 250/505] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:30.253239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b83a2-f7fd-4dc2-99b2-cf0d55ed4990 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:30.276387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:30.309320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 256/506] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:30.310172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b72cc1-aef7-4129-95b8-9cbc61291e70 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:30.365542 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b83a2-f7fd-4dc2-99b2-cf0d55ed4990 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:30.401424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf496d1-a9cd-49f4-9cd2-c0e5a50fa09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:30.402039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf496d1-a9cd-49f4-9cd2-c0e5a50fa09a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:30.402377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf496d1-a9cd-49f4-9cd2-c0e5a50fa09a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:30.402696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf496d1-a9cd-49f4-9cd2-c0e5a50fa09a None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:30.403444 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf496d1-a9cd-49f4-9cd2-c0e5a50fa09a None admin] RBAC: Adding query filter params (name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:30.403803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf496d1-a9cd-49f4-9cd2-c0e5a50fa09a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:30.405258 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf496d1-a9cd-49f4-9cd2-c0e5a50fa09a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:30.413541 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b72cc1-aef7-4129-95b8-9cbc61291e70 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:30.418514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 251/507] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:30.418514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf496d1-a9cd-49f4-9cd2-c0e5a50fa09a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:30.453330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4848979-aadc-4bb0-866d-4325349ec3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:30.454114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4848979-aadc-4bb0-866d-4325349ec3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:30.454522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4848979-aadc-4bb0-866d-4325349ec3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:30.454888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4848979-aadc-4bb0-866d-4325349ec3e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:30.460299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:30.466639 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4848979-aadc-4bb0-866d-4325349ec3e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:30.468126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4848979-aadc-4bb0-866d-4325349ec3e8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-d4848979-aadc-4bb0-866d-4325349ec3e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:30.483025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:30.484148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:30.485360 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:30.485360 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:30.485360 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:30.488968 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 257/508] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:30.489799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4848979-aadc-4bb0-866d-4325349ec3e8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:30.516271 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:30.530614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf496d1-a9cd-49f4-9cd2-c0e5a50fa09a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:30.566667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d629c-54e5-4111-80aa-ccdf35ca92b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:30.566876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d629c-54e5-4111-80aa-ccdf35ca92b7 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:30.567146 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d629c-54e5-4111-80aa-ccdf35ca92b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:30.567447 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d629c-54e5-4111-80aa-ccdf35ca92b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:30.568117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d629c-54e5-4111-80aa-ccdf35ca92b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:30.568414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d629c-54e5-4111-80aa-ccdf35ca92b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:30.569791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d629c-54e5-4111-80aa-ccdf35ca92b7 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:30.582697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 252/509] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:30.583514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d629c-54e5-4111-80aa-ccdf35ca92b7 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:30.597140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4848979-aadc-4bb0-866d-4325349ec3e8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:30.630493 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbac4d-e4c8-4fc3-95cb-023abea51c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:30.631086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbac4d-e4c8-4fc3-95cb-023abea51c74 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:30.631358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbac4d-e4c8-4fc3-95cb-023abea51c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:30.631622 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbac4d-e4c8-4fc3-95cb-023abea51c74 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:30.632386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdbac4d-e4c8-4fc3-95cb-023abea51c74 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:30.632691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdbac4d-e4c8-4fc3-95cb-023abea51c74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:30.634122 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdbac4d-e4c8-4fc3-95cb-023abea51c74 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:30.651282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 258/510] 10.222.0.62 () {56 vars in 1139 bytes} [Thu Sep 1 16:23:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:30.652268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbac4d-e4c8-4fc3-95cb-023abea51c74 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:30.679706 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:30.683542 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d629c-54e5-4111-80aa-ccdf35ca92b7 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:30.718046 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f1302-d0fe-4151-90b3-500e15cbe44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:30.718728 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f1302-d0fe-4151-90b3-500e15cbe44f None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:30.718938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f1302-d0fe-4151-90b3-500e15cbe44f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:30.719195 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f1302-d0fe-4151-90b3-500e15cbe44f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:30.729140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f1302-d0fe-4151-90b3-500e15cbe44f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:30.730827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f1302-d0fe-4151-90b3-500e15cbe44f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:30.735750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-b07f1302-d0fe-4151-90b3-500e15cbe44f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:30.744900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:30.745977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:30.746827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:30.746827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:30.746827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:30.746827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 253/511] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:30.746827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f1302-d0fe-4151-90b3-500e15cbe44f None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:30.761650 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbac4d-e4c8-4fc3-95cb-023abea51c74 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:30.799873 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35c703-5b52-42a4-9eea-64fc81069f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:30.800507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35c703-5b52-42a4-9eea-64fc81069f10 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:30.800782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35c703-5b52-42a4-9eea-64fc81069f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:30.801046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35c703-5b52-42a4-9eea-64fc81069f10 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:30.801698 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35c703-5b52-42a4-9eea-64fc81069f10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:30.801992 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35c703-5b52-42a4-9eea-64fc81069f10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:30.803387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35c703-5b52-42a4-9eea-64fc81069f10 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:30.815504 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 259/512] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:30.820163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35c703-5b52-42a4-9eea-64fc81069f10 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:30.850395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f1302-d0fe-4151-90b3-500e15cbe44f None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:30.884365 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7075d-dc79-4092-8f99-301b5b8824bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:30.885941 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7075d-dc79-4092-8f99-301b5b8824bc None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:30.886226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7075d-dc79-4092-8f99-301b5b8824bc None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:30.886495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7075d-dc79-4092-8f99-301b5b8824bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:30.899178 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7075d-dc79-4092-8f99-301b5b8824bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:30.900854 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7075d-dc79-4092-8f99-301b5b8824bc None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-b5d7075d-dc79-4092-8f99-301b5b8824bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:30.916454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:30.917987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:30.918885 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:30.918885 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:30.918885 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:30.918885 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 254/513] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:30.918885 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7075d-dc79-4092-8f99-301b5b8824bc None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:30.921602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:30.927161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35c703-5b52-42a4-9eea-64fc81069f10 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:30.962099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b6436-8543-400c-8935-897e46831ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:30.962756 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b6436-8543-400c-8935-897e46831ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:30.963024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b6436-8543-400c-8935-897e46831ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:30.963284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b6436-8543-400c-8935-897e46831ad1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:30.969935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b6436-8543-400c-8935-897e46831ad1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:30.971383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b6436-8543-400c-8935-897e46831ad1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2e4b6436-8543-400c-8935-897e46831ad1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:30.984366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:30.985357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:30.986226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:30.986226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:30.986226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:30.986226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:30.986226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:30.986226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:30.986226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:30.986226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:30.986226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 260/514] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:30.986675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b6436-8543-400c-8935-897e46831ad1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:30.989024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:31.026697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7075d-dc79-4092-8f99-301b5b8824bc None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:31.059988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e217c2b-71d8-4bfe-8ac0-c23f1656b1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:31.060592 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e217c2b-71d8-4bfe-8ac0-c23f1656b1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:31.060871 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e217c2b-71d8-4bfe-8ac0-c23f1656b1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:31.061141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e217c2b-71d8-4bfe-8ac0-c23f1656b1c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:31.085948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b6436-8543-400c-8935-897e46831ad1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:31.121026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe1f0e-814e-49c5-977d-94b4cfb23402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:31.121644 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe1f0e-814e-49c5-977d-94b4cfb23402 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:31.121947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe1f0e-814e-49c5-977d-94b4cfb23402 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:31.122213 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe1f0e-814e-49c5-977d-94b4cfb23402 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:31.122848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe1f0e-814e-49c5-977d-94b4cfb23402 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:31.123146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe1f0e-814e-49c5-977d-94b4cfb23402 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:31.124565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe1f0e-814e-49c5-977d-94b4cfb23402 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:31.127581 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e217c2b-71d8-4bfe-8ac0-c23f1656b1c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:31.129073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e217c2b-71d8-4bfe-8ac0-c23f1656b1c1 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:31.130878 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:31.138396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 261/515] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:31.139135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe1f0e-814e-49c5-977d-94b4cfb23402 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:31.200496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:31.232327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 255/516] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:31.233179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e217c2b-71d8-4bfe-8ac0-c23f1656b1c1 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:31.239255 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe1f0e-814e-49c5-977d-94b4cfb23402 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:31.275419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ceeca6-77e7-48f9-9ead-966d86ec1191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:31.276090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ceeca6-77e7-48f9-9ead-966d86ec1191 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:31.276343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ceeca6-77e7-48f9-9ead-966d86ec1191 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:31.276616 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ceeca6-77e7-48f9-9ead-966d86ec1191 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:31.284146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ceeca6-77e7-48f9-9ead-966d86ec1191 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:31.285596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ceeca6-77e7-48f9-9ead-966d86ec1191 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e6ceeca6-77e7-48f9-9ead-966d86ec1191 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:31.297450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:31.298347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:31.299228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:31.299228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:31.299228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:31.299228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:31.299228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:31.299228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:31.299228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:31.299228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:31.299228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:23:31.299228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:31.299228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 262/517] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:31.299766 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ceeca6-77e7-48f9-9ead-966d86ec1191 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:31.338698 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e217c2b-71d8-4bfe-8ac0-c23f1656b1c1 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:31.339994 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:31.373394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f510f-4e92-4963-868e-f02ac064b9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:31.373987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f510f-4e92-4963-868e-f02ac064b9bd None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:31.374256 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f510f-4e92-4963-868e-f02ac064b9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:31.374517 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f510f-4e92-4963-868e-f02ac064b9bd None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:31.375144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f510f-4e92-4963-868e-f02ac064b9bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:31.375438 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f510f-4e92-4963-868e-f02ac064b9bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:31.376830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f510f-4e92-4963-868e-f02ac064b9bd None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:31.387844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 256/518] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:31.388647 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f510f-4e92-4963-868e-f02ac064b9bd None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:31.401113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ceeca6-77e7-48f9-9ead-966d86ec1191 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:31.422940 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:31.435195 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42028f4-ae26-4a68-a297-8384faf91b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:31.435807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42028f4-ae26-4a68-a297-8384faf91b5f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:31.436134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42028f4-ae26-4a68-a297-8384faf91b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:31.436403 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42028f4-ae26-4a68-a297-8384faf91b5f None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:31.437092 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42028f4-ae26-4a68-a297-8384faf91b5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:31.437393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42028f4-ae26-4a68-a297-8384faf91b5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:31.438822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42028f4-ae26-4a68-a297-8384faf91b5f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:31.465236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 263/519] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:31.466034 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42028f4-ae26-4a68-a297-8384faf91b5f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:31.498276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f510f-4e92-4963-868e-f02ac064b9bd None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:31.530434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753485c1-d5f5-48a3-99c1-8a4478130ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:31.531057 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753485c1-d5f5-48a3-99c1-8a4478130ddc None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:31.531334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753485c1-d5f5-48a3-99c1-8a4478130ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:31.531598 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753485c1-d5f5-48a3-99c1-8a4478130ddc None admin] PATH_INFO: `/v3/users/glance` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:31.538766 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753485c1-d5f5-48a3-99c1-8a4478130ddc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:31.540239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753485c1-d5f5-48a3-99c1-8a4478130ddc None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-753485c1-d5f5-48a3-99c1-8a4478130ddc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:31.551369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:31.552297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:31.553168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:31.553168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:31.553168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:31.553168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:31.553168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:31.553168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:31.553168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:31.553168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:31.553168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:23:31.553168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:31.553168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 257/520] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:31.553709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753485c1-d5f5-48a3-99c1-8a4478130ddc None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:31.574547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42028f4-ae26-4a68-a297-8384faf91b5f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:31.578817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:31.610356 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f3927-a308-405c-af0f-7e0854dadd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:31.610982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f3927-a308-405c-af0f-7e0854dadd4f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:31.611253 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f3927-a308-405c-af0f-7e0854dadd4f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:31.611517 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f3927-a308-405c-af0f-7e0854dadd4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:31.621739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f3927-a308-405c-af0f-7e0854dadd4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:31.623223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f3927-a308-405c-af0f-7e0854dadd4f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-da7f3927-a308-405c-af0f-7e0854dadd4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:31.637679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:31.638785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:31.639658 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:31.639658 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:31.639658 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:31.639658 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 264/521] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:31.639658 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f3927-a308-405c-af0f-7e0854dadd4f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:31.661825 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753485c1-d5f5-48a3-99c1-8a4478130ddc None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:31.675038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:31.698698 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfec51-6904-46af-95b4-b37e0493d72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:31.699303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfec51-6904-46af-95b4-b37e0493d72b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:31.699612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfec51-6904-46af-95b4-b37e0493d72b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:31.699875 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfec51-6904-46af-95b4-b37e0493d72b None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:31.700623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abfec51-6904-46af-95b4-b37e0493d72b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:31.700923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abfec51-6904-46af-95b4-b37e0493d72b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:31.702353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abfec51-6904-46af-95b4-b37e0493d72b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:31.739166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 258/522] 10.222.0.62 () {56 vars in 1137 bytes} [Thu Sep 1 16:23:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:31.740665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfec51-6904-46af-95b4-b37e0493d72b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:31.742083 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f3927-a308-405c-af0f-7e0854dadd4f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:31.778497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6c270-acfc-40bf-9a0c-3d0ceebdee99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:31.779095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6c270-acfc-40bf-9a0c-3d0ceebdee99 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:31.779394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6c270-acfc-40bf-9a0c-3d0ceebdee99 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:31.779660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6c270-acfc-40bf-9a0c-3d0ceebdee99 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:31.780328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6c270-acfc-40bf-9a0c-3d0ceebdee99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:31.781742 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6c270-acfc-40bf-9a0c-3d0ceebdee99 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:31.803191 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 265/523] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:23:31 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:23:31.804018 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6c270-acfc-40bf-9a0c-3d0ceebdee99 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:31.828326 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:31.852226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfec51-6904-46af-95b4-b37e0493d72b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:31.891152 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09887270-acba-423b-93fb-e66e59cbf655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:31.891772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09887270-acba-423b-93fb-e66e59cbf655 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:31.892082 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09887270-acba-423b-93fb-e66e59cbf655 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:31.892355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09887270-acba-423b-93fb-e66e59cbf655 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:31.893093 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09887270-acba-423b-93fb-e66e59cbf655 None admin] RBAC: Adding query filter params (role.id=fc3db8095f31464ca4c378645d1d078b, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=2912c02ff6db4cbc97c80cd923238136) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:31.893400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09887270-acba-423b-93fb-e66e59cbf655 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:31.894899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09887270-acba-423b-93fb-e66e59cbf655 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:31.901241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 259/524] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:31 2022] GET /identity/v3/role_assignments?user.id=2912c02ff6db4cbc97c80cd923238136&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=fc3db8095f31464ca4c378645d1d078b => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:31.902101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09887270-acba-423b-93fb-e66e59cbf655 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:31.933419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:31.951174 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6c270-acfc-40bf-9a0c-3d0ceebdee99 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:32.005025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf7b9f-4464-4ef3-b493-c4eda70d02e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:32.005025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf7b9f-4464-4ef3-b493-c4eda70d02e7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:32.005025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf7b9f-4464-4ef3-b493-c4eda70d02e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:32.005352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf7b9f-4464-4ef3-b493-c4eda70d02e7 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:32.006022 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf7b9f-4464-4ef3-b493-c4eda70d02e7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:32.006328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf7b9f-4464-4ef3-b493-c4eda70d02e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:32.007798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf7b9f-4464-4ef3-b493-c4eda70d02e7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:32.036084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 266/525] 10.222.0.62 () {56 vars in 1137 bytes} [Thu Sep 1 16:23:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:32.037050 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf7b9f-4464-4ef3-b493-c4eda70d02e7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:32.050643 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09887270-acba-423b-93fb-e66e59cbf655 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:32.098204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4646557-e6c4-4387-a40c-203d5826409c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:32.099217 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4646557-e6c4-4387-a40c-203d5826409c None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:32.099660 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4646557-e6c4-4387-a40c-203d5826409c None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:32.100179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4646557-e6c4-4387-a40c-203d5826409c None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:32.100821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4646557-e6c4-4387-a40c-203d5826409c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:32.101119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4646557-e6c4-4387-a40c-203d5826409c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:32.102503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4646557-e6c4-4387-a40c-203d5826409c None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:32.126406 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 260/526] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:32.127738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4646557-e6c4-4387-a40c-203d5826409c None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:32.133667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:32.173905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf7b9f-4464-4ef3-b493-c4eda70d02e7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:32.211103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7b3e-ee57-4cbb-ab74-e017756159d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:32.212101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7b3e-ee57-4cbb-ab74-e017756159d3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:32.212630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7b3e-ee57-4cbb-ab74-e017756159d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:32.213088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7b3e-ee57-4cbb-ab74-e017756159d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:32.232648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:32.272746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4646557-e6c4-4387-a40c-203d5826409c None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:32.289241 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a7b3e-ee57-4cbb-ab74-e017756159d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:32.291191 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a7b3e-ee57-4cbb-ab74-e017756159d3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:32.320824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe9f34-dcf4-4b17-af51-a9ae43b54991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:32.321681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe9f34-dcf4-4b17-af51-a9ae43b54991 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:32.322293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe9f34-dcf4-4b17-af51-a9ae43b54991 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:32.322570 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe9f34-dcf4-4b17-af51-a9ae43b54991 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:32.406662 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe9f34-dcf4-4b17-af51-a9ae43b54991 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:32.408473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe9f34-dcf4-4b17-af51-a9ae43b54991 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:32.410364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 267/527] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:32.411306 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:32.411625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7b3e-ee57-4cbb-ab74-e017756159d3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:32.476639 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:32.519861 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 261/528] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:32.520758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe9f34-dcf4-4b17-af51-a9ae43b54991 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:32.545464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7b3e-ee57-4cbb-ab74-e017756159d3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:32.590913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1453a2-f451-4b11-94f8-373c12bf3e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:32.591987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1453a2-f451-4b11-94f8-373c12bf3e0e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:32.592467 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1453a2-f451-4b11-94f8-373c12bf3e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:32.592950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1453a2-f451-4b11-94f8-373c12bf3e0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:32.601871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1453a2-f451-4b11-94f8-373c12bf3e0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:32.604058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1453a2-f451-4b11-94f8-373c12bf3e0e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-cc1453a2-f451-4b11-94f8-373c12bf3e0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:32.618576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:32.619464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:32.620447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:32.620447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:32.620447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:32.620447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:32.620447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:32.620447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:32.620447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:32.620447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:32.627920 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 268/529] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:32.627920 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1453a2-f451-4b11-94f8-373c12bf3e0e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:32.638829 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe9f34-dcf4-4b17-af51-a9ae43b54991 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:32.643107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:32.686095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02965f50-55c1-4938-98af-5ee6d009a020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:32.687328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02965f50-55c1-4938-98af-5ee6d009a020 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:32.688079 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02965f50-55c1-4938-98af-5ee6d009a020 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:32.688963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02965f50-55c1-4938-98af-5ee6d009a020 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:32.702527 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02965f50-55c1-4938-98af-5ee6d009a020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:32.705356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02965f50-55c1-4938-98af-5ee6d009a020 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-02965f50-55c1-4938-98af-5ee6d009a020 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:32.723238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:32.724253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:32.725075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:32.725075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:32.725075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:32.725075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 262/530] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:32.725299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02965f50-55c1-4938-98af-5ee6d009a020 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:32.746134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1453a2-f451-4b11-94f8-373c12bf3e0e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:32.758009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:32.793725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa113be3-39bd-43d2-9b1b-4fca97c18570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:32.794844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa113be3-39bd-43d2-9b1b-4fca97c18570 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:32.795284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa113be3-39bd-43d2-9b1b-4fca97c18570 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:32.796036 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa113be3-39bd-43d2-9b1b-4fca97c18570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:32.811497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa113be3-39bd-43d2-9b1b-4fca97c18570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:32.814122 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa113be3-39bd-43d2-9b1b-4fca97c18570 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-aa113be3-39bd-43d2-9b1b-4fca97c18570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:32.843947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:32.845089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:32.846151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:32.846151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:32.846151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:32.848144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 269/531] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:32.849183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa113be3-39bd-43d2-9b1b-4fca97c18570 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:32.854494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02965f50-55c1-4938-98af-5ee6d009a020 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:32.892524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05805f-f7ce-4e33-8077-9ced9dabc5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:32.893533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05805f-f7ce-4e33-8077-9ced9dabc5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:32.893906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05805f-f7ce-4e33-8077-9ced9dabc5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:32.894269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05805f-f7ce-4e33-8077-9ced9dabc5b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:32.895057 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05805f-f7ce-4e33-8077-9ced9dabc5b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:32.895470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05805f-f7ce-4e33-8077-9ced9dabc5b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:32.897259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd05805f-f7ce-4e33-8077-9ced9dabc5b7 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:32.921816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 263/532] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:32.921816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05805f-f7ce-4e33-8077-9ced9dabc5b7 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:32.936673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:32.974428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa113be3-39bd-43d2-9b1b-4fca97c18570 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:33.011555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f4107-a651-46f9-a053-9e13ee89d342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:33.012734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f4107-a651-46f9-a053-9e13ee89d342 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:33.013259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f4107-a651-46f9-a053-9e13ee89d342 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:33.013707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f4107-a651-46f9-a053-9e13ee89d342 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:33.014555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f4107-a651-46f9-a053-9e13ee89d342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:33.015038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f4107-a651-46f9-a053-9e13ee89d342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:33.016828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f4107-a651-46f9-a053-9e13ee89d342 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:33.031087 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05805f-f7ce-4e33-8077-9ced9dabc5b7 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:33.032189 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 270/533] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:33.033399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f4107-a651-46f9-a053-9e13ee89d342 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:33.057944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:33.070843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a4232-694a-46af-b38a-f1efa52b7a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:33.071535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a4232-694a-46af-b38a-f1efa52b7a62 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:33.072572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a4232-694a-46af-b38a-f1efa52b7a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:33.072922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a4232-694a-46af-b38a-f1efa52b7a62 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:33.073662 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a4232-694a-46af-b38a-f1efa52b7a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:33.073989 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a4232-694a-46af-b38a-f1efa52b7a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:33.075535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a4232-694a-46af-b38a-f1efa52b7a62 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:33.088440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 264/534] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:33.089448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a4232-694a-46af-b38a-f1efa52b7a62 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:33.146809 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f4107-a651-46f9-a053-9e13ee89d342 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:33.172492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:33.193818 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075ff93-3710-4038-a089-f87aeeb0419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:33.194773 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075ff93-3710-4038-a089-f87aeeb0419d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:33.195238 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075ff93-3710-4038-a089-f87aeeb0419d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:33.195689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075ff93-3710-4038-a089-f87aeeb0419d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:33.196014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a4232-694a-46af-b38a-f1efa52b7a62 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:33.197057 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075ff93-3710-4038-a089-f87aeeb0419d None admin] RBAC: Adding query filter params (role.id=fc3db8095f31464ca4c378645d1d078b, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=0718a95189434a5ea901c40bc868bea4) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:33.197564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075ff93-3710-4038-a089-f87aeeb0419d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:33.199431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075ff93-3710-4038-a089-f87aeeb0419d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:33.205440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 271/535] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:33 2022] GET /identity/v3/role_assignments?user.id=0718a95189434a5ea901c40bc868bea4&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=fc3db8095f31464ca4c378645d1d078b => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:33.212121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075ff93-3710-4038-a089-f87aeeb0419d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:33.234502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e185421-048e-4f02-a7dc-0066e8600245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:33.235113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e185421-048e-4f02-a7dc-0066e8600245 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:33.235393 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e185421-048e-4f02-a7dc-0066e8600245 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:33.235659 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e185421-048e-4f02-a7dc-0066e8600245 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:33.247765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e185421-048e-4f02-a7dc-0066e8600245 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:33.247765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e185421-048e-4f02-a7dc-0066e8600245 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-2e185421-048e-4f02-a7dc-0066e8600245 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:33.260911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:33.261820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:33.262652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:33.262652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:33.262652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:33.262652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:33.262652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:33.262652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:33.262652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:33.262652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:33.262652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 265/536] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:33.268171 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e185421-048e-4f02-a7dc-0066e8600245 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:33.421568 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e185421-048e-4f02-a7dc-0066e8600245 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:33.430544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:33.457973 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075ff93-3710-4038-a089-f87aeeb0419d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:33.498352 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95475a3-7d4f-44c9-918e-29142139a5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:33.498964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95475a3-7d4f-44c9-918e-29142139a5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:33.499236 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95475a3-7d4f-44c9-918e-29142139a5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:33.499502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95475a3-7d4f-44c9-918e-29142139a5c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:33.500220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95475a3-7d4f-44c9-918e-29142139a5c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:33.500521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95475a3-7d4f-44c9-918e-29142139a5c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:33.502799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95475a3-7d4f-44c9-918e-29142139a5c1 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:33.512277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:33.526298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a8e6e-f0e0-49e1-95b8-d4e8c450d94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:33.526937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a8e6e-f0e0-49e1-95b8-d4e8c450d94c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:33.527211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a8e6e-f0e0-49e1-95b8-d4e8c450d94c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:33.527476 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a8e6e-f0e0-49e1-95b8-d4e8c450d94c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:33.528028 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 266/537] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:33.530665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95475a3-7d4f-44c9-918e-29142139a5c1 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:33.538891 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a8e6e-f0e0-49e1-95b8-d4e8c450d94c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:33.540440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a8e6e-f0e0-49e1-95b8-d4e8c450d94c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-289a8e6e-f0e0-49e1-95b8-d4e8c450d94c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:33.564540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:33.565527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:33.566354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:33.566354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:33.566354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:33.566354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:33.566354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:33.566354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:33.566354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:33.566354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:33.566354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 272/538] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:33.573838 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a8e6e-f0e0-49e1-95b8-d4e8c450d94c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:33.733166 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a8e6e-f0e0-49e1-95b8-d4e8c450d94c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:33.761904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95475a3-7d4f-44c9-918e-29142139a5c1 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:33.779815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c96b8-3d08-4b3c-b579-d43a7fe28d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:33.781117 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c96b8-3d08-4b3c-b579-d43a7fe28d19 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:33.781582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c96b8-3d08-4b3c-b579-d43a7fe28d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:33.782035 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c96b8-3d08-4b3c-b579-d43a7fe28d19 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:33.782943 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c96b8-3d08-4b3c-b579-d43a7fe28d19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:33.783428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c96b8-3d08-4b3c-b579-d43a7fe28d19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:33.787364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c96b8-3d08-4b3c-b579-d43a7fe28d19 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:33.812281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 273/539] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:33.819606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c96b8-3d08-4b3c-b579-d43a7fe28d19 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:33.826808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:33.840449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024433ac-a287-48ad-9f60-4340cc0fbd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:33.841118 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024433ac-a287-48ad-9f60-4340cc0fbd79 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:33.841392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024433ac-a287-48ad-9f60-4340cc0fbd79 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:33.841700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024433ac-a287-48ad-9f60-4340cc0fbd79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:33.842493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024433ac-a287-48ad-9f60-4340cc0fbd79 None admin] RBAC: Adding query filter params (role.id=fc3db8095f31464ca4c378645d1d078b, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=6160b373ace44ccfaf557b557343bf32) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:33.842796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024433ac-a287-48ad-9f60-4340cc0fbd79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:33.844320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024433ac-a287-48ad-9f60-4340cc0fbd79 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:33.857089 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 267/540] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:33 2022] GET /identity/v3/role_assignments?user.id=6160b373ace44ccfaf557b557343bf32&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=fc3db8095f31464ca4c378645d1d078b => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:33.865458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacabe3-0355-4639-ab24-eed474a5e3fb None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:33.865733 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacabe3-0355-4639-ab24-eed474a5e3fb None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:33.866311 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacabe3-0355-4639-ab24-eed474a5e3fb None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:33.867634 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 268/541] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:33.875985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46982c76-c67b-4a3e-bad9-aaa9771d586d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:33.878689 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46982c76-c67b-4a3e-bad9-aaa9771d586d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:33.879107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46982c76-c67b-4a3e-bad9-aaa9771d586d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:33.902555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-46982c76-c67b-4a3e-bad9-aaa9771d586d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:33.903947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-46982c76-c67b-4a3e-bad9-aaa9771d586d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:33.972328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:33.979957 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c96b8-3d08-4b3c-b579-d43a7fe28d19 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:33.986892 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 269/542] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:23:33.988462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e1c75-0030-428e-9fd8-cf00cf23b910 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:33.988851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e1c75-0030-428e-9fd8-cf00cf23b910 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:33.989215 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e1c75-0030-428e-9fd8-cf00cf23b910 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:33.995090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 270/543] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:33.997127 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1485fd-a792-486f-befd-33808020cf2a None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:33.997477 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1485fd-a792-486f-befd-33808020cf2a None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:33.997805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1485fd-a792-486f-befd-33808020cf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:34.017035 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-7e1485fd-a792-486f-befd-33808020cf2a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:34.018307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7e1485fd-a792-486f-befd-33808020cf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:34.024699 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a2a38-aad2-4986-a50a-1af35917e246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:34.026030 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a2a38-aad2-4986-a50a-1af35917e246 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:34.026314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a2a38-aad2-4986-a50a-1af35917e246 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:34.026587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a2a38-aad2-4986-a50a-1af35917e246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:34.027947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a2a38-aad2-4986-a50a-1af35917e246 None admin] RBAC: Adding query filter params (role.id=fc3db8095f31464ca4c378645d1d078b, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=3cfe36a14a11493397e4018b773d3cc4) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:34.028266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a2a38-aad2-4986-a50a-1af35917e246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:34.030175 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334a2a38-aad2-4986-a50a-1af35917e246 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:34.037691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 274/544] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:33 2022] GET /identity/v3/role_assignments?user.id=3cfe36a14a11493397e4018b773d3cc4&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=fc3db8095f31464ca4c378645d1d078b => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:34.101506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 271/545] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:23:34.291521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a2a38-aad2-4986-a50a-1af35917e246 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:34.376122 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:34.419858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a2a38-aad2-4986-a50a-1af35917e246 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:34.465536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc8b0b-a1b9-461f-9a7d-e9178572c9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:34.466196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc8b0b-a1b9-461f-9a7d-e9178572c9df None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:34.466470 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc8b0b-a1b9-461f-9a7d-e9178572c9df None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:34.466737 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc8b0b-a1b9-461f-9a7d-e9178572c9df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:34.482434 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc8b0b-a1b9-461f-9a7d-e9178572c9df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-6ccc8b0b-a1b9-461f-9a7d-e9178572c9df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:34.487864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:34.488774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:34.490180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:34.490180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:34.490180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:34.490180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:34.490180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:34.490180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:34.490180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:34.490180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:34.490180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:34.490180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:34.490180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:34.490180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:34.490180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 275/546] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:34.495331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1485fd-a792-486f-befd-33808020cf2a None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:34.496224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc8b0b-a1b9-461f-9a7d-e9178572c9df None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:34.646114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc8b0b-a1b9-461f-9a7d-e9178572c9df None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:34.678014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:34.687679 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1485fd-a792-486f-befd-33808020cf2a None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:34.690716 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1889d5-cb6f-4f98-9e5a-0ae84e71fc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:34.691361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1889d5-cb6f-4f98-9e5a-0ae84e71fc87 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:34.691632 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1889d5-cb6f-4f98-9e5a-0ae84e71fc87 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:34.691918 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1889d5-cb6f-4f98-9e5a-0ae84e71fc87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:34.714264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1889d5-cb6f-4f98-9e5a-0ae84e71fc87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ef1889d5-cb6f-4f98-9e5a-0ae84e71fc87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:34.721515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:34.722377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:34.723181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:34.723181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:34.723181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:34.723181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:34.723181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:34.723181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:34.723181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:34.723181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:34.723181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:34.723181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:34.723181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:34.723181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:34.723181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 276/547] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:23:34.729362 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:34.732189 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1889d5-cb6f-4f98-9e5a-0ae84e71fc87 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:34.777184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c06a9-e826-441e-b9b5-c77268769a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:34.777822 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c06a9-e826-441e-b9b5-c77268769a8e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:34.778095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c06a9-e826-441e-b9b5-c77268769a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:34.778359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c06a9-e826-441e-b9b5-c77268769a8e None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:34.779027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c06a9-e826-441e-b9b5-c77268769a8e None admin] RBAC: Adding query filter params (name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:34.779321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c06a9-e826-441e-b9b5-c77268769a8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:34.780744 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c06a9-e826-441e-b9b5-c77268769a8e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:34.806329 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 272/548] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:34.811950 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c06a9-e826-441e-b9b5-c77268769a8e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:34.851666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1889d5-cb6f-4f98-9e5a-0ae84e71fc87 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:34.886875 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044dd5b-4dc1-4623-b31c-bb52de757024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:34.887496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044dd5b-4dc1-4623-b31c-bb52de757024 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:34.887775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044dd5b-4dc1-4623-b31c-bb52de757024 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:34.888093 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044dd5b-4dc1-4623-b31c-bb52de757024 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:34.888761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044dd5b-4dc1-4623-b31c-bb52de757024 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:34.889062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044dd5b-4dc1-4623-b31c-bb52de757024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:34.890587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044dd5b-4dc1-4623-b31c-bb52de757024 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:34.899578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 277/549] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:34.905475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044dd5b-4dc1-4623-b31c-bb52de757024 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:35.012712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:35.053597 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044dd5b-4dc1-4623-b31c-bb52de757024 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:35.066066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c06a9-e826-441e-b9b5-c77268769a8e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:35.088980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46ccdb-8b78-48de-92af-a5ecace1e720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:35.089666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46ccdb-8b78-48de-92af-a5ecace1e720 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:35.089946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46ccdb-8b78-48de-92af-a5ecace1e720 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:35.090216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46ccdb-8b78-48de-92af-a5ecace1e720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:35.101753 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46ccdb-8b78-48de-92af-a5ecace1e720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:35.103289 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46ccdb-8b78-48de-92af-a5ecace1e720 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-cc46ccdb-8b78-48de-92af-a5ecace1e720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:35.118544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:35.120074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:35.120913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:35.120913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:35.120913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:35.120913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 278/550] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:35.131175 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46ccdb-8b78-48de-92af-a5ecace1e720 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:35.139066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf2175-ab04-40c6-aeb6-4d6cd84a931d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:35.140000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf2175-ab04-40c6-aeb6-4d6cd84a931d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:35.140497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf2175-ab04-40c6-aeb6-4d6cd84a931d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:35.140968 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf2175-ab04-40c6-aeb6-4d6cd84a931d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:35.156514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf2175-ab04-40c6-aeb6-4d6cd84a931d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:35.158443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf2175-ab04-40c6-aeb6-4d6cd84a931d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-47bf2175-ab04-40c6-aeb6-4d6cd84a931d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:35.183355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:35.184361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:35.185392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:35.185392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:35.185392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:35.196898 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 273/551] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:35.202195 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf2175-ab04-40c6-aeb6-4d6cd84a931d None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:35.267489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:35.344750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46ccdb-8b78-48de-92af-a5ecace1e720 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:35.348482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:35.399488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf2175-ab04-40c6-aeb6-4d6cd84a931d None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:35.420335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1e603-6b4a-4db4-be7f-75c57c2dfa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:35.421263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1e603-6b4a-4db4-be7f-75c57c2dfa82 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:35.421728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1e603-6b4a-4db4-be7f-75c57c2dfa82 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:35.422201 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1e603-6b4a-4db4-be7f-75c57c2dfa82 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:35.423061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1e603-6b4a-4db4-be7f-75c57c2dfa82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:35.423553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1e603-6b4a-4db4-be7f-75c57c2dfa82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:35.425151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1e603-6b4a-4db4-be7f-75c57c2dfa82 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:35.449046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 279/552] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:35.455964 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454badf-ce11-4e3b-8b11-785f856b7179 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:35.456463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454badf-ce11-4e3b-8b11-785f856b7179 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:35.456920 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454badf-ce11-4e3b-8b11-785f856b7179 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:35.459404 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 280/553] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:35.460301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454badf-ce11-4e3b-8b11-785f856b7179 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:35.465517 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452678a-035c-411f-a920-97f6889097d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:35.466429 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452678a-035c-411f-a920-97f6889097d8 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:35.467008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452678a-035c-411f-a920-97f6889097d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:35.467509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452678a-035c-411f-a920-97f6889097d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:35.468425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452678a-035c-411f-a920-97f6889097d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:35.468892 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452678a-035c-411f-a920-97f6889097d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:35.470268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452678a-035c-411f-a920-97f6889097d8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:35.488231 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 274/554] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:35.489502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb223183-3380-4ce8-8dfe-1053824ea420 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:35.489782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb223183-3380-4ce8-8dfe-1053824ea420 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:35.490061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb223183-3380-4ce8-8dfe-1053824ea420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:35.509481 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-eb223183-3380-4ce8-8dfe-1053824ea420 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:35.510608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-eb223183-3380-4ce8-8dfe-1053824ea420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:35.577510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454badf-ce11-4e3b-8b11-785f856b7179 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:35.587419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 275/555] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:35.588416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb223183-3380-4ce8-8dfe-1053824ea420 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:35.614624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:35.617711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9ee5f-fa80-4a7d-9795-ec0134923957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:35.619385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9ee5f-fa80-4a7d-9795-ec0134923957 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:35.619930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9ee5f-fa80-4a7d-9795-ec0134923957 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:35.620415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9ee5f-fa80-4a7d-9795-ec0134923957 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:35.628683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9ee5f-fa80-4a7d-9795-ec0134923957 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:35.630457 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9ee5f-fa80-4a7d-9795-ec0134923957 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-28a9ee5f-fa80-4a7d-9795-ec0134923957 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:35.643660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:35.644647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:35.645567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:35.645567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:35.645567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:35.645567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:35.645567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:35.645567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:35.645567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:35.645567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:35.645567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:23:35.645567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:35.648389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 281/556] 10.222.0.62 () {56 vars in 1093 bytes} [Thu Sep 1 16:23:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:35.658404 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9ee5f-fa80-4a7d-9795-ec0134923957 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:35.710986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:35.739984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb223183-3380-4ce8-8dfe-1053824ea420 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:35.808828 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a9221-30a1-4f33-9e02-adf661aa58bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:35.809467 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a9221-30a1-4f33-9e02-adf661aa58bc None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:35.809733 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a9221-30a1-4f33-9e02-adf661aa58bc None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:35.809995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a9221-30a1-4f33-9e02-adf661aa58bc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:35.819774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a9221-30a1-4f33-9e02-adf661aa58bc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:35.828745 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a9221-30a1-4f33-9e02-adf661aa58bc None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-4e8a9221-30a1-4f33-9e02-adf661aa58bc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:35.847019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:35.848480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:35.849999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:35.849999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:35.849999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:35.849999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:35.849999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:35.849999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:35.849999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:35.849999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:35.849999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:23:35.849999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:35.849999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 276/557] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:23:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:35.858528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a9221-30a1-4f33-9e02-adf661aa58bc None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:35.874644 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9ee5f-fa80-4a7d-9795-ec0134923957 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:35.915623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc133e50-2e9a-41ef-94fe-5d1c42d1deaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:35.916291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc133e50-2e9a-41ef-94fe-5d1c42d1deaf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:35.916562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc133e50-2e9a-41ef-94fe-5d1c42d1deaf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:35.916829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc133e50-2e9a-41ef-94fe-5d1c42d1deaf None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:35.917553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc133e50-2e9a-41ef-94fe-5d1c42d1deaf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:35.917849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc133e50-2e9a-41ef-94fe-5d1c42d1deaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:35.919264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc133e50-2e9a-41ef-94fe-5d1c42d1deaf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:35.939580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 282/558] 10.222.0.62 () {56 vars in 1133 bytes} [Thu Sep 1 16:23:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:35.940656 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc133e50-2e9a-41ef-94fe-5d1c42d1deaf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:35.976157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:35.994690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a9221-30a1-4f33-9e02-adf661aa58bc None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:36.029796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb505bd3-7b28-4b45-9d0c-de8d1c5e168a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:36.030405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb505bd3-7b28-4b45-9d0c-de8d1c5e168a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:36.031012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb505bd3-7b28-4b45-9d0c-de8d1c5e168a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:36.031012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb505bd3-7b28-4b45-9d0c-de8d1c5e168a None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:36.031643 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb505bd3-7b28-4b45-9d0c-de8d1c5e168a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:36.031978 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb505bd3-7b28-4b45-9d0c-de8d1c5e168a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:36.033686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb505bd3-7b28-4b45-9d0c-de8d1c5e168a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:36.037936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:36.058539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 277/559] 10.222.0.62 () {56 vars in 1135 bytes} [Thu Sep 1 16:23:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:36.059528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb505bd3-7b28-4b45-9d0c-de8d1c5e168a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:36.061764 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc133e50-2e9a-41ef-94fe-5d1c42d1deaf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:36.098991 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d80e31-dead-48ae-a0cc-5e07015036c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:36.100056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d80e31-dead-48ae-a0cc-5e07015036c3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:36.100534 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d80e31-dead-48ae-a0cc-5e07015036c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:36.101014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d80e31-dead-48ae-a0cc-5e07015036c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:36.112487 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d80e31-dead-48ae-a0cc-5e07015036c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e8d80e31-dead-48ae-a0cc-5e07015036c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:36.117837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:36.118981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:36.120481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:36.120481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:36.120481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:36.120481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:36.120481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:36.120481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:36.120481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:36.120481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:36.120481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:36.120481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:36.120481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:36.120481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:36.124297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 283/560] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:36.125919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d80e31-dead-48ae-a0cc-5e07015036c3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:36.172194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb505bd3-7b28-4b45-9d0c-de8d1c5e168a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:36.214678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe23c3f-ade5-420b-a083-7ae5a2b6d67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:36.215445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe23c3f-ade5-420b-a083-7ae5a2b6d67d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:36.215577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe23c3f-ade5-420b-a083-7ae5a2b6d67d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:36.215842 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe23c3f-ade5-420b-a083-7ae5a2b6d67d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:36.226231 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe23c3f-ade5-420b-a083-7ae5a2b6d67d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:36.227611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe23c3f-ade5-420b-a083-7ae5a2b6d67d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:36.229951 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:36.241917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d80e31-dead-48ae-a0cc-5e07015036c3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-4fe23c3f-ade5-420b-a083-7ae5a2b6d67d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:36.244101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:36.245216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:36.246279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:36.246279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:36.246279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:36.246279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 278/561] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:36.246279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe23c3f-ade5-420b-a083-7ae5a2b6d67d None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:36.278371 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a654680-9313-412c-865c-484e8e78f530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:36.279000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654680-9313-412c-865c-484e8e78f530 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:36.279253 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654680-9313-412c-865c-484e8e78f530 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:36.279519 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a654680-9313-412c-865c-484e8e78f530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:36.289881 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:36.346768 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a654680-9313-412c-865c-484e8e78f530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:36.348301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a654680-9313-412c-865c-484e8e78f530 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:36.348550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe23c3f-ade5-420b-a083-7ae5a2b6d67d None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:36.381119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f994d-7a85-4885-9545-393497d4539b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:36.381716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f994d-7a85-4885-9545-393497d4539b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:36.381988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f994d-7a85-4885-9545-393497d4539b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:36.382254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f994d-7a85-4885-9545-393497d4539b None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:36.382906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f994d-7a85-4885-9545-393497d4539b None admin] RBAC: Adding query filter params (name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:36.383208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f994d-7a85-4885-9545-393497d4539b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:36.384885 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f994d-7a85-4885-9545-393497d4539b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:36.392685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 279/562] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:36.393912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60d2b4-a67e-4ec8-bb3e-b590da36241b None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:36.394200 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60d2b4-a67e-4ec8-bb3e-b590da36241b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:36.394470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60d2b4-a67e-4ec8-bb3e-b590da36241b None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:36.395820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 280/563] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:36.401120 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07787175-8eb0-4bd3-a0d4-7a5f5498fed6 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:36.401428 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07787175-8eb0-4bd3-a0d4-7a5f5498fed6 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:36.401697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07787175-8eb0-4bd3-a0d4-7a5f5498fed6 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:36.402908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 281/564] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:36.403637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07787175-8eb0-4bd3-a0d4-7a5f5498fed6 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:36.447380 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 284/565] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:36.448242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a654680-9313-412c-865c-484e8e78f530 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:36.463449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:36.490417 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:36.508017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07787175-8eb0-4bd3-a0d4-7a5f5498fed6 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:36.542035 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845fcd0-c063-4a88-b1ee-36ec64f8b5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:36.542632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845fcd0-c063-4a88-b1ee-36ec64f8b5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:36.542900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845fcd0-c063-4a88-b1ee-36ec64f8b5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:36.543164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845fcd0-c063-4a88-b1ee-36ec64f8b5e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:36.543791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845fcd0-c063-4a88-b1ee-36ec64f8b5e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:36.544124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845fcd0-c063-4a88-b1ee-36ec64f8b5e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:36.545494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845fcd0-c063-4a88-b1ee-36ec64f8b5e4 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:36.547547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a654680-9313-412c-865c-484e8e78f530 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:36.557695 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 282/566] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:36.558869 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a75d6-583a-45f2-8b18-634a0c20387c None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:36.559147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a75d6-583a-45f2-8b18-634a0c20387c None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:36.559438 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a75d6-583a-45f2-8b18-634a0c20387c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:36.577402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-2f4a75d6-583a-45f2-8b18-634a0c20387c None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:36.578590 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2f4a75d6-583a-45f2-8b18-634a0c20387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:36.583027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0047c-4d4b-436e-9a6b-462ff2f4d3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:36.583660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0047c-4d4b-436e-9a6b-462ff2f4d3ca None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:36.583993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0047c-4d4b-436e-9a6b-462ff2f4d3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:36.584276 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0047c-4d4b-436e-9a6b-462ff2f4d3ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:36.594525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0047c-4d4b-436e-9a6b-462ff2f4d3ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:36.595986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0047c-4d4b-436e-9a6b-462ff2f4d3ca None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-66e0047c-4d4b-436e-9a6b-462ff2f4d3ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:36.610411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:36.611367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:36.612749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:36.612749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:36.612749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:36.612749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 285/567] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:36.613067 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd68d0-ee75-43d9-a3af-db70ce8014be None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:36.613357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd68d0-ee75-43d9-a3af-db70ce8014be None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:36.613626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd68d0-ee75-43d9-a3af-db70ce8014be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:36.631480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-afbd68d0-ee75-43d9-a3af-db70ce8014be None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:36.632737 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-afbd68d0-ee75-43d9-a3af-db70ce8014be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:36.651469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 283/568] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:36.652312 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a75d6-583a-45f2-8b18-634a0c20387c None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:36.700205 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 286/569] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:36.701370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd68d0-ee75-43d9-a3af-db70ce8014be None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:36.734717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:36.737325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:36.823340 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd68d0-ee75-43d9-a3af-db70ce8014be None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:36.823906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a75d6-583a-45f2-8b18-634a0c20387c None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:36.860394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d4a48-2622-4f06-b02c-457286ed2bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:36.861240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d4a48-2622-4f06-b02c-457286ed2bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:36.861612 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d4a48-2622-4f06-b02c-457286ed2bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:36.861975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d4a48-2622-4f06-b02c-457286ed2bf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:36.865661 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350e7af-ce33-438c-9917-2d4835121a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:36.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e7af-ce33-438c-9917-2d4835121a2a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:36.866574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e7af-ce33-438c-9917-2d4835121a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:36.866841 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350e7af-ce33-438c-9917-2d4835121a2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:36.870368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d4a48-2622-4f06-b02c-457286ed2bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:36.871925 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d4a48-2622-4f06-b02c-457286ed2bf2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:36.877510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350e7af-ce33-438c-9917-2d4835121a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:36.878969 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350e7af-ce33-438c-9917-2d4835121a2a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9f0d4a48-2622-4f06-b02c-457286ed2bf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:36.884098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:36.885397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:36.886746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:36.886746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:36.886746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:36.886746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:36.886746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:36.886746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:36.886746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:36.886746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:36.886746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 287/570] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:36.886746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d4a48-2622-4f06-b02c-457286ed2bf2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-4350e7af-ce33-438c-9917-2d4835121a2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:36.894331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:36.896363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:36.898424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:36.898424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:36.898424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:36.901618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 284/571] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:36.901618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350e7af-ce33-438c-9917-2d4835121a2a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:36.973274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:36.995384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d4a48-2622-4f06-b02c-457286ed2bf2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:37.001829 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:37.005786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350e7af-ce33-438c-9917-2d4835121a2a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:37.029104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedc718-9070-40a3-80a3-1ccf6c31d470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:37.029635 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedc718-9070-40a3-80a3-1ccf6c31d470 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:37.029905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedc718-9070-40a3-80a3-1ccf6c31d470 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:37.030168 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cedc718-9070-40a3-80a3-1ccf6c31d470 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:37.030797 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cedc718-9070-40a3-80a3-1ccf6c31d470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:37.031090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cedc718-9070-40a3-80a3-1ccf6c31d470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:37.032502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cedc718-9070-40a3-80a3-1ccf6c31d470 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:37.039510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc379677-a86a-486b-bde2-6bd939d777fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:37.040149 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc379677-a86a-486b-bde2-6bd939d777fa None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:37.040428 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc379677-a86a-486b-bde2-6bd939d777fa None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:37.040755 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc379677-a86a-486b-bde2-6bd939d777fa None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:37.041488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc379677-a86a-486b-bde2-6bd939d777fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:37.041790 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc379677-a86a-486b-bde2-6bd939d777fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:37.043217 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc379677-a86a-486b-bde2-6bd939d777fa None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:37.044140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 288/572] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:37.044928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedc718-9070-40a3-80a3-1ccf6c31d470 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:37.055536 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 285/573] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:37.056387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc379677-a86a-486b-bde2-6bd939d777fa None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:37.149505 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cedc718-9070-40a3-80a3-1ccf6c31d470 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:37.156941 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc379677-a86a-486b-bde2-6bd939d777fa None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:37.183724 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e8d85-cc17-4ef6-9ae8-242e7ccf1456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:37.184369 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e8d85-cc17-4ef6-9ae8-242e7ccf1456 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:37.184637 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e8d85-cc17-4ef6-9ae8-242e7ccf1456 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:37.184900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e8d85-cc17-4ef6-9ae8-242e7ccf1456 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:37.185532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e8d85-cc17-4ef6-9ae8-242e7ccf1456 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:37.185827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e8d85-cc17-4ef6-9ae8-242e7ccf1456 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:37.187204 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e8d85-cc17-4ef6-9ae8-242e7ccf1456 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:37.191637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b94c2b-895f-4653-a239-685ae7f62018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:37.192286 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b94c2b-895f-4653-a239-685ae7f62018 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:37.192555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b94c2b-895f-4653-a239-685ae7f62018 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:37.192814 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b94c2b-895f-4653-a239-685ae7f62018 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:37.198530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 289/574] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:37.199319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e8d85-cc17-4ef6-9ae8-242e7ccf1456 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:37.201815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:37.207409 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b94c2b-895f-4653-a239-685ae7f62018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-a4b94c2b-895f-4653-a239-685ae7f62018 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:37.212244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:37.213460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:37.214749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:37.214749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:37.214749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:37.214749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:37.214749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:37.214749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:37.214749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:37.214749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:37.214749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:37.214749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:37.214749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:37.214749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:37.214749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 286/575] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:37.214749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b94c2b-895f-4653-a239-685ae7f62018 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:37.232667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:37.303841 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e8d85-cc17-4ef6-9ae8-242e7ccf1456 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:37.311928 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b94c2b-895f-4653-a239-685ae7f62018 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:37.337031 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02329911-b8e2-4daa-93ce-4e2de87a3e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:37.337775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02329911-b8e2-4daa-93ce-4e2de87a3e96 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:37.338046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02329911-b8e2-4daa-93ce-4e2de87a3e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:37.338320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02329911-b8e2-4daa-93ce-4e2de87a3e96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:37.345521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecc8bc-71e1-4888-bad7-9158477a6ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:37.346149 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecc8bc-71e1-4888-bad7-9158477a6ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:37.346426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecc8bc-71e1-4888-bad7-9158477a6ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:37.346692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecc8bc-71e1-4888-bad7-9158477a6ad1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:37.348123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02329911-b8e2-4daa-93ce-4e2de87a3e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-02329911-b8e2-4daa-93ce-4e2de87a3e96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:37.353386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:37.354518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:37.355358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:37.355358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:37.355358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:37.355358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:37.355358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:37.355358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:37.355358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:37.355358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:37.355358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:37.355358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:37.355358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:37.355358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:37.356051 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecc8bc-71e1-4888-bad7-9158477a6ad1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:37.356122 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 290/576] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:37.356204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecc8bc-71e1-4888-bad7-9158477a6ad1 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:37.356283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02329911-b8e2-4daa-93ce-4e2de87a3e96 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-d3ecc8bc-71e1-4888-bad7-9158477a6ad1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:37.369601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:37.375857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:37.377611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:37.377611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:37.377611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:37.377611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:37.377611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:37.377611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:37.377611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:37.377611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:37.377611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:23:37.377611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:37.377611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 287/577] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:37.377611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecc8bc-71e1-4888-bad7-9158477a6ad1 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:37.419552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:37.452964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:37.459758 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02329911-b8e2-4daa-93ce-4e2de87a3e96 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:37.475154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecc8bc-71e1-4888-bad7-9158477a6ad1 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:37.494039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940576a-b67c-4f2e-814c-9e751f5380c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:37.494630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940576a-b67c-4f2e-814c-9e751f5380c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:37.494911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940576a-b67c-4f2e-814c-9e751f5380c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:37.495200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940576a-b67c-4f2e-814c-9e751f5380c0 None admin] PATH_INFO: `/v3/projects/425048ffd13448cd8fe815333a9b798c/users/2912c02ff6db4cbc97c80cd923238136/roles/fc3db8095f31464ca4c378645d1d078b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:37.508388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cda98-e2aa-427c-9c7a-1cba814e6a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:37.509005 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cda98-e2aa-427c-9c7a-1cba814e6a26 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:37.509334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cda98-e2aa-427c-9c7a-1cba814e6a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:37.509601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cda98-e2aa-427c-9c7a-1cba814e6a26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:37.509697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940576a-b67c-4f2e-814c-9e751f5380c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=425048ffd13448cd8fe815333a9b798c, user_id=2912c02ff6db4cbc97c80cd923238136, role_id=fc3db8095f31464ca4c378645d1d078b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:37.511107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940576a-b67c-4f2e-814c-9e751f5380c0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:37.516163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cda98-e2aa-427c-9c7a-1cba814e6a26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:37.517584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cda98-e2aa-427c-9c7a-1cba814e6a26 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:37.524288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 291/578] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:23:37 2022] PUT /identity/v3/projects/425048ffd13448cd8fe815333a9b798c/users/2912c02ff6db4cbc97c80cd923238136/roles/fc3db8095f31464ca4c378645d1d078b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:23:37.525416 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940576a-b67c-4f2e-814c-9e751f5380c0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-fa2cda98-e2aa-427c-9c7a-1cba814e6a26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:37.529893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:37.530975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:37.537061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:37.537061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:37.537061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:37.537061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:37.537061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:37.537061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:37.537061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:37.537061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:37.537061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 288/579] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:37.537061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cda98-e2aa-427c-9c7a-1cba814e6a26 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:37.659321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cda98-e2aa-427c-9c7a-1cba814e6a26 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:37.663632 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940576a-b67c-4f2e-814c-9e751f5380c0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:37.680451 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:37.701991 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c48cb-4424-4774-a564-8949568a91ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:37.702607 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c48cb-4424-4774-a564-8949568a91ea None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:37.702887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c48cb-4424-4774-a564-8949568a91ea None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:37.703158 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c48cb-4424-4774-a564-8949568a91ea None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:37.703821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c48cb-4424-4774-a564-8949568a91ea None admin] RBAC: Adding query filter params (name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:37.704177 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c48cb-4424-4774-a564-8949568a91ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:37.705588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c48cb-4424-4774-a564-8949568a91ea None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:37.705873 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3621524-1c24-4e1d-90a7-f84f183108dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:37.706490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621524-1c24-4e1d-90a7-f84f183108dd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:37.706766 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621524-1c24-4e1d-90a7-f84f183108dd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:37.707508 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621524-1c24-4e1d-90a7-f84f183108dd None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:37.708533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3621524-1c24-4e1d-90a7-f84f183108dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:37.708840 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3621524-1c24-4e1d-90a7-f84f183108dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:37.710183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3621524-1c24-4e1d-90a7-f84f183108dd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:37.711314 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:37.716061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 289/580] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:37.717660 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c48cb-4424-4774-a564-8949568a91ea None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:37.724061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 292/581] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:37.724061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3621524-1c24-4e1d-90a7-f84f183108dd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:37.829222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c48cb-4424-4774-a564-8949568a91ea None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:37.838852 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3621524-1c24-4e1d-90a7-f84f183108dd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:37.866213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032236a-7822-4c2a-80b6-205da107eb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:37.866830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032236a-7822-4c2a-80b6-205da107eb95 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:37.867110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032236a-7822-4c2a-80b6-205da107eb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:37.867380 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032236a-7822-4c2a-80b6-205da107eb95 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:37.868147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032236a-7822-4c2a-80b6-205da107eb95 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:37.868453 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032236a-7822-4c2a-80b6-205da107eb95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:37.870095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032236a-7822-4c2a-80b6-205da107eb95 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:37.873757 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b270f10-d62a-45f7-8422-e4ee59c5efc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:37.874380 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b270f10-d62a-45f7-8422-e4ee59c5efc9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:37.874656 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b270f10-d62a-45f7-8422-e4ee59c5efc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:37.874928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b270f10-d62a-45f7-8422-e4ee59c5efc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:37.875616 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b270f10-d62a-45f7-8422-e4ee59c5efc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:37.875945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b270f10-d62a-45f7-8422-e4ee59c5efc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:37.877370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b270f10-d62a-45f7-8422-e4ee59c5efc9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:37.889202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 290/582] 10.222.0.62 () {56 vars in 1139 bytes} [Thu Sep 1 16:23:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:37.890234 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 293/583] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:37.890994 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032236a-7822-4c2a-80b6-205da107eb95 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:37.891116 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b270f10-d62a-45f7-8422-e4ee59c5efc9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:37.931993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:37.960652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:38.002309 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b270f10-d62a-45f7-8422-e4ee59c5efc9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:38.004278 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032236a-7822-4c2a-80b6-205da107eb95 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:38.038209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76f864-c698-4425-8b5e-231bd042e4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:38.038815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f864-c698-4425-8b5e-231bd042e4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:38.039114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f864-c698-4425-8b5e-231bd042e4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:38.039357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f864-c698-4425-8b5e-231bd042e4a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:38.044470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d86ca0-1d1c-4967-bae2-a4986283c902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:38.045349 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d86ca0-1d1c-4967-bae2-a4986283c902 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:38.045799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d86ca0-1d1c-4967-bae2-a4986283c902 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:38.046635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d86ca0-1d1c-4967-bae2-a4986283c902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:38.049643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76f864-c698-4425-8b5e-231bd042e4a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:38.051097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76f864-c698-4425-8b5e-231bd042e4a1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:38.059260 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d86ca0-1d1c-4967-bae2-a4986283c902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:38.060744 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d86ca0-1d1c-4967-bae2-a4986283c902 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-3c76f864-c698-4425-8b5e-231bd042e4a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:38.066358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:38.067279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:38.068189 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:38.068189 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:38.068189 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:38.072203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 294/584] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:38.072921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76f864-c698-4425-8b5e-231bd042e4a1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-62d86ca0-1d1c-4967-bae2-a4986283c902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:38.077285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:38.078305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:38.079495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:38.079495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:38.079495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:38.079495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 291/585] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:38.079726 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d86ca0-1d1c-4967-bae2-a4986283c902 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:38.167083 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:38.182593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76f864-c698-4425-8b5e-231bd042e4a1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:38.198551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d86ca0-1d1c-4967-bae2-a4986283c902 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:38.211346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:38.223603 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe81fd-f585-44ba-9308-ed3202fce0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:38.224366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe81fd-f585-44ba-9308-ed3202fce0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:38.224646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe81fd-f585-44ba-9308-ed3202fce0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:38.224923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe81fd-f585-44ba-9308-ed3202fce0c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:38.235836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe81fd-f585-44ba-9308-ed3202fce0c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:38.237325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe81fd-f585-44ba-9308-ed3202fce0c2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:38.240539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16332f42-a05d-47b7-af46-d15509fd3ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:38.241353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16332f42-a05d-47b7-af46-d15509fd3ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:38.241808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16332f42-a05d-47b7-af46-d15509fd3ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:38.242276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16332f42-a05d-47b7-af46-d15509fd3ee0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:38.243185 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16332f42-a05d-47b7-af46-d15509fd3ee0 None admin] RBAC: Adding query filter params (role.id=fc3db8095f31464ca4c378645d1d078b, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=56f9bc7262b64bbab0f670576c6bdfdd) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:38.244114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16332f42-a05d-47b7-af46-d15509fd3ee0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:38.246324 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16332f42-a05d-47b7-af46-d15509fd3ee0 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-5ffe81fd-f585-44ba-9308-ed3202fce0c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:38.252302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:38.253265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:38.254615 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:38.254615 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:38.254615 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:38.258146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 295/586] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:38.258930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe81fd-f585-44ba-9308-ed3202fce0c2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:38.262129 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 292/587] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:38 2022] GET /identity/v3/role_assignments?user.id=56f9bc7262b64bbab0f670576c6bdfdd&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=fc3db8095f31464ca4c378645d1d078b => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:38.263253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16332f42-a05d-47b7-af46-d15509fd3ee0 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:38.392735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16332f42-a05d-47b7-af46-d15509fd3ee0 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:38.400024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe81fd-f585-44ba-9308-ed3202fce0c2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:38.435949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655ae9b-9974-4913-a7a6-34c5c4b8f986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:38.436613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655ae9b-9974-4913-a7a6-34c5c4b8f986 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:38.436894 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655ae9b-9974-4913-a7a6-34c5c4b8f986 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:38.437179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655ae9b-9974-4913-a7a6-34c5c4b8f986 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:38.437797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655ae9b-9974-4913-a7a6-34c5c4b8f986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:38.438141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655ae9b-9974-4913-a7a6-34c5c4b8f986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:38.439151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25422a8d-1e6e-422c-b302-95813beb22d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:38.439725 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655ae9b-9974-4913-a7a6-34c5c4b8f986 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:38.440096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25422a8d-1e6e-422c-b302-95813beb22d5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:38.440525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25422a8d-1e6e-422c-b302-95813beb22d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:38.440936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25422a8d-1e6e-422c-b302-95813beb22d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:38.441784 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25422a8d-1e6e-422c-b302-95813beb22d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:38.442291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25422a8d-1e6e-422c-b302-95813beb22d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:38.443831 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25422a8d-1e6e-422c-b302-95813beb22d5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:38.452665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:38.455956 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 293/588] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:38.457384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655ae9b-9974-4913-a7a6-34c5c4b8f986 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:38.465575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 296/589] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:38.466775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25422a8d-1e6e-422c-b302-95813beb22d5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:38.505587 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:38.585003 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25422a8d-1e6e-422c-b302-95813beb22d5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:38.591243 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655ae9b-9974-4913-a7a6-34c5c4b8f986 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:38.622943 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ceb3f-3477-435f-b85f-dfbffff19524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:38.623748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ceb3f-3477-435f-b85f-dfbffff19524 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:38.624141 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ceb3f-3477-435f-b85f-dfbffff19524 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:38.624521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ceb3f-3477-435f-b85f-dfbffff19524 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:38.633547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ceb3f-3477-435f-b85f-dfbffff19524 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:38.636087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ceb3f-3477-435f-b85f-dfbffff19524 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:38.637581 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed428c92-cdb0-4980-bf9e-0ff3ececf51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:38.638471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed428c92-cdb0-4980-bf9e-0ff3ececf51b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:38.639184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed428c92-cdb0-4980-bf9e-0ff3ececf51b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:38.639668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed428c92-cdb0-4980-bf9e-0ff3ececf51b None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:38.641387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed428c92-cdb0-4980-bf9e-0ff3ececf51b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:38.641882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed428c92-cdb0-4980-bf9e-0ff3ececf51b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:38.643548 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed428c92-cdb0-4980-bf9e-0ff3ececf51b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b52ceb3f-3477-435f-b85f-dfbffff19524 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:38.652712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:38.653966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:38.655148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:38.655148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:38.655148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:38.655148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:38.655148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:38.655148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:38.655148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:38.655148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:38.655148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:23:38.655148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:38.661995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 297/590] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:38.661995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ceb3f-3477-435f-b85f-dfbffff19524 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:38.676392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 294/591] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:38.676392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed428c92-cdb0-4980-bf9e-0ff3ececf51b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:38.740483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:38.785254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ceb3f-3477-435f-b85f-dfbffff19524 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:38.804944 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:38.817371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed428c92-cdb0-4980-bf9e-0ff3ececf51b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:38.823578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3b261-8246-4c42-9157-0f627a0604c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:38.824222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3b261-8246-4c42-9157-0f627a0604c6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:38.824490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3b261-8246-4c42-9157-0f627a0604c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:38.824750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3b261-8246-4c42-9157-0f627a0604c6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:38.834486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3b261-8246-4c42-9157-0f627a0604c6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:38.835936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3b261-8246-4c42-9157-0f627a0604c6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-05d3b261-8246-4c42-9157-0f627a0604c6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:38.855164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:38.856158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:38.857107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:38.857107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:38.857107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:38.857107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:38.857107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:38.857107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:38.857107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:38.857107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:38.857107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:23:38.857107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:38.858408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 298/592] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:38.859429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3b261-8246-4c42-9157-0f627a0604c6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:38.869860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f73334-46ca-4035-8eb3-72a73d89a8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:38.870533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f73334-46ca-4035-8eb3-72a73d89a8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:38.870805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f73334-46ca-4035-8eb3-72a73d89a8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:38.871083 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f73334-46ca-4035-8eb3-72a73d89a8a9 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:38.871899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f73334-46ca-4035-8eb3-72a73d89a8a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:38.872227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f73334-46ca-4035-8eb3-72a73d89a8a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:38.873879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f73334-46ca-4035-8eb3-72a73d89a8a9 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:38.898808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 295/593] 10.222.0.62 () {56 vars in 1137 bytes} [Thu Sep 1 16:23:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:38.899718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f73334-46ca-4035-8eb3-72a73d89a8a9 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:38.978218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3b261-8246-4c42-9157-0f627a0604c6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:38.997248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:39.016443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2d464-1f0e-41e2-9a62-87492f041755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:39.017100 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2d464-1f0e-41e2-9a62-87492f041755 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:39.017396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2d464-1f0e-41e2-9a62-87492f041755 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:39.017635 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2d464-1f0e-41e2-9a62-87492f041755 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:39.025034 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2d464-1f0e-41e2-9a62-87492f041755 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:39.026383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2d464-1f0e-41e2-9a62-87492f041755 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:39.035347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f73334-46ca-4035-8eb3-72a73d89a8a9 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2ac2d464-1f0e-41e2-9a62-87492f041755 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:39.038631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:39.039499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:39.041294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:39.041294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:39.041294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:39.041294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:39.041294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:39.041294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:39.041294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:39.041294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:39.043844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 299/594] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:39.044823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2d464-1f0e-41e2-9a62-87492f041755 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:39.081158 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b28833-5aaa-47d6-ba5a-9a63cf69996a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:39.081765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b28833-5aaa-47d6-ba5a-9a63cf69996a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:39.082039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b28833-5aaa-47d6-ba5a-9a63cf69996a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:39.082671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b28833-5aaa-47d6-ba5a-9a63cf69996a None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:39.083402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b28833-5aaa-47d6-ba5a-9a63cf69996a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:39.083697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b28833-5aaa-47d6-ba5a-9a63cf69996a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:39.085274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b28833-5aaa-47d6-ba5a-9a63cf69996a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:39.108415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 296/595] 10.222.0.62 () {56 vars in 1137 bytes} [Thu Sep 1 16:23:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:39.109364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b28833-5aaa-47d6-ba5a-9a63cf69996a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:39.122702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:39.181701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2d464-1f0e-41e2-9a62-87492f041755 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:39.216551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b28833-5aaa-47d6-ba5a-9a63cf69996a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:39.227775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1a51b-dd00-48c5-8c3c-b69d75cc9d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:39.229255 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1a51b-dd00-48c5-8c3c-b69d75cc9d21 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:39.229844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1a51b-dd00-48c5-8c3c-b69d75cc9d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:39.230256 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1a51b-dd00-48c5-8c3c-b69d75cc9d21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:39.245789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1a51b-dd00-48c5-8c3c-b69d75cc9d21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:39.248058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1a51b-dd00-48c5-8c3c-b69d75cc9d21 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:39.253493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea806dc-2348-43e3-9265-4afc7ffd1992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:39.254216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea806dc-2348-43e3-9265-4afc7ffd1992 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:39.254578 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea806dc-2348-43e3-9265-4afc7ffd1992 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:39.254937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea806dc-2348-43e3-9265-4afc7ffd1992 None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:39.255719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea806dc-2348-43e3-9265-4afc7ffd1992 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:39.256185 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea806dc-2348-43e3-9265-4afc7ffd1992 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:39.257708 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea806dc-2348-43e3-9265-4afc7ffd1992 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-eab1a51b-dd00-48c5-8c3c-b69d75cc9d21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:39.269284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:39.270440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:39.271506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:39.271506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:39.271506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:39.273236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 300/597] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:39.274212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1a51b-dd00-48c5-8c3c-b69d75cc9d21 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:39.279861 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 297/597] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:39.280932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea806dc-2348-43e3-9265-4afc7ffd1992 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:39.315422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:39.374457 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:39.394459 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1a51b-dd00-48c5-8c3c-b69d75cc9d21 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:39.417861 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea806dc-2348-43e3-9265-4afc7ffd1992 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:39.431172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b4e4b-a4fc-4f53-a7db-52df4572648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:39.431728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b4e4b-a4fc-4f53-a7db-52df4572648c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:39.432132 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b4e4b-a4fc-4f53-a7db-52df4572648c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:39.432488 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b4e4b-a4fc-4f53-a7db-52df4572648c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:39.443777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b4e4b-a4fc-4f53-a7db-52df4572648c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:39.445390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b4e4b-a4fc-4f53-a7db-52df4572648c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-af6b4e4b-a4fc-4f53-a7db-52df4572648c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:39.460985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:39.462206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:39.463490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:39.463490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:39.463490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:39.464060 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a441e-f6b9-4370-808d-cc6af1a362f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:39.464890 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a441e-f6b9-4370-808d-cc6af1a362f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:39.465339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a441e-f6b9-4370-808d-cc6af1a362f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:39.465841 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a441e-f6b9-4370-808d-cc6af1a362f7 None admin] PATH_INFO: `/v3/projects/425048ffd13448cd8fe815333a9b798c/users/0718a95189434a5ea901c40bc868bea4/roles/fc3db8095f31464ca4c378645d1d078b` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:39.470226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 301/598] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:39.470880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b4e4b-a4fc-4f53-a7db-52df4572648c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:39.485031 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a441e-f6b9-4370-808d-cc6af1a362f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=425048ffd13448cd8fe815333a9b798c, user_id=0718a95189434a5ea901c40bc868bea4, role_id=fc3db8095f31464ca4c378645d1d078b)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:39.487059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a441e-f6b9-4370-808d-cc6af1a362f7 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:39.508353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 298/599] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:23:39 2022] PUT /identity/v3/projects/425048ffd13448cd8fe815333a9b798c/users/0718a95189434a5ea901c40bc868bea4/roles/fc3db8095f31464ca4c378645d1d078b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:23:39.509600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a441e-f6b9-4370-808d-cc6af1a362f7 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:39.577011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:39.598377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b4e4b-a4fc-4f53-a7db-52df4572648c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:39.640246 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3d70a-8ace-4d86-b96c-d89a8d6fcd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:39.640811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3d70a-8ace-4d86-b96c-d89a8d6fcd25 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:39.641095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3d70a-8ace-4d86-b96c-d89a8d6fcd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:39.641373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3d70a-8ace-4d86-b96c-d89a8d6fcd25 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:39.642024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3d70a-8ace-4d86-b96c-d89a8d6fcd25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:39.642331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3d70a-8ace-4d86-b96c-d89a8d6fcd25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:39.644277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3d70a-8ace-4d86-b96c-d89a8d6fcd25 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:39.654243 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a441e-f6b9-4370-808d-cc6af1a362f7 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:39.660819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 302/600] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:39.662291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ad5d4-77e4-4b1b-8d83-3094227a4529 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:39.662582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ad5d4-77e4-4b1b-8d83-3094227a4529 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:39.662852 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ad5d4-77e4-4b1b-8d83-3094227a4529 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:39.664785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 303/601] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:39.668954 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ad5d4-77e4-4b1b-8d83-3094227a4529 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:39.679156 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:39.702136 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0dce91-a6d2-424a-b981-1a71b1ca3024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:39.702860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dce91-a6d2-424a-b981-1a71b1ca3024 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:39.703138 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dce91-a6d2-424a-b981-1a71b1ca3024 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:39.703410 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dce91-a6d2-424a-b981-1a71b1ca3024 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:39.704084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0dce91-a6d2-424a-b981-1a71b1ca3024 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:39.704385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0dce91-a6d2-424a-b981-1a71b1ca3024 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:39.705920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0dce91-a6d2-424a-b981-1a71b1ca3024 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:39.724591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 299/602] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:39.724591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ec023-ebef-4461-a28e-ec84e6cc111d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:39.724591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ec023-ebef-4461-a28e-ec84e6cc111d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:39.724591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ec023-ebef-4461-a28e-ec84e6cc111d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:39.741321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-aa3ec023-ebef-4461-a28e-ec84e6cc111d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:39.742541 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-aa3ec023-ebef-4461-a28e-ec84e6cc111d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:39.794533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ad5d4-77e4-4b1b-8d83-3094227a4529 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:39.835548 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 300/603] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:39.836789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ec023-ebef-4461-a28e-ec84e6cc111d None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:39.840156 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f9481-5089-45cd-bb59-a832f1835e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:39.841411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f9481-5089-45cd-bb59-a832f1835e5f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:39.841411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f9481-5089-45cd-bb59-a832f1835e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:39.841605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f9481-5089-45cd-bb59-a832f1835e5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:39.861348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f9481-5089-45cd-bb59-a832f1835e5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:39.862837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f9481-5089-45cd-bb59-a832f1835e5f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-4d7f9481-5089-45cd-bb59-a832f1835e5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:39.882197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:39.883167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:39.884102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:39.884102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:39.884102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:39.884102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:39.884102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:39.884102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:39.884102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:39.884102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:39.888113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 304/604] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:39.889315 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f9481-5089-45cd-bb59-a832f1835e5f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:39.892866 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:39.960890 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ec023-ebef-4461-a28e-ec84e6cc111d None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:39.974820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:40.025233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c5e42-1976-4917-b8ba-922275b254ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:40.026149 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c5e42-1976-4917-b8ba-922275b254ac None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:40.026423 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c5e42-1976-4917-b8ba-922275b254ac None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:40.026687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c5e42-1976-4917-b8ba-922275b254ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:40.040770 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c5e42-1976-4917-b8ba-922275b254ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:40.042273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c5e42-1976-4917-b8ba-922275b254ac None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-a67c5e42-1976-4917-b8ba-922275b254ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:40.068110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:40.069055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:40.069942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:40.069942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:40.069942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:40.069942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:40.069942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:40.069942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:40.069942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:40.069942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:40.070830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 301/605] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:40.076476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c5e42-1976-4917-b8ba-922275b254ac None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:40.098984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f9481-5089-45cd-bb59-a832f1835e5f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:40.154669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9920d-4dc2-43dc-8a80-7f1575c0c527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:40.155294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9920d-4dc2-43dc-8a80-7f1575c0c527 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:40.155569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9920d-4dc2-43dc-8a80-7f1575c0c527 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:40.155877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9920d-4dc2-43dc-8a80-7f1575c0c527 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:40.156602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9920d-4dc2-43dc-8a80-7f1575c0c527 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:40.156900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9920d-4dc2-43dc-8a80-7f1575c0c527 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:40.158918 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9920d-4dc2-43dc-8a80-7f1575c0c527 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:40.173987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 305/606] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:40.182462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9920d-4dc2-43dc-8a80-7f1575c0c527 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:40.254284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c5e42-1976-4917-b8ba-922275b254ac None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:40.260108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:40.289680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f186d-6e8b-413f-90f7-13f682485117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:40.290202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f186d-6e8b-413f-90f7-13f682485117 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:40.290521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f186d-6e8b-413f-90f7-13f682485117 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:40.290798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f186d-6e8b-413f-90f7-13f682485117 None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:40.291483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f186d-6e8b-413f-90f7-13f682485117 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:40.291782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f186d-6e8b-413f-90f7-13f682485117 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:40.293246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f186d-6e8b-413f-90f7-13f682485117 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:40.302860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9920d-4dc2-43dc-8a80-7f1575c0c527 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:40.305226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 302/607] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:40.306581 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591b475-7d37-456d-9dbe-539618083b33 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:40.306867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591b475-7d37-456d-9dbe-539618083b33 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:40.307130 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591b475-7d37-456d-9dbe-539618083b33 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:40.308605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 303/608] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:40.313150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591b475-7d37-456d-9dbe-539618083b33 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:40.343818 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:40.347501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b79cc-1643-4e8f-af72-93c926e84e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:40.348417 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b79cc-1643-4e8f-af72-93c926e84e68 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:40.349311 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b79cc-1643-4e8f-af72-93c926e84e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:40.349827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b79cc-1643-4e8f-af72-93c926e84e68 None admin] PATH_INFO: `/v3/projects/425048ffd13448cd8fe815333a9b798c/users/3cfe36a14a11493397e4018b773d3cc4/roles/fc3db8095f31464ca4c378645d1d078b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:40.365668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b79cc-1643-4e8f-af72-93c926e84e68 None admin] RBAC: Authorizing `identity:create_grant(project_id=425048ffd13448cd8fe815333a9b798c, user_id=3cfe36a14a11493397e4018b773d3cc4, role_id=fc3db8095f31464ca4c378645d1d078b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:40.367226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b79cc-1643-4e8f-af72-93c926e84e68 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:40.391919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 306/609] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:23:40 2022] PUT /identity/v3/projects/425048ffd13448cd8fe815333a9b798c/users/3cfe36a14a11493397e4018b773d3cc4/roles/fc3db8095f31464ca4c378645d1d078b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:23:40.391919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b79cc-1643-4e8f-af72-93c926e84e68 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:40.440278 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591b475-7d37-456d-9dbe-539618083b33 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:40.500283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4f198-479b-4b2a-990f-e016178f5c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:40.501313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4f198-479b-4b2a-990f-e016178f5c4d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:40.501656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4f198-479b-4b2a-990f-e016178f5c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:40.501992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4f198-479b-4b2a-990f-e016178f5c4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:40.513479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b79cc-1643-4e8f-af72-93c926e84e68 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:40.519207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4f198-479b-4b2a-990f-e016178f5c4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-a2e4f198-479b-4b2a-990f-e016178f5c4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:40.526068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:40.527188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:40.528182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:40.528182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:40.528182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:40.528182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:40.528182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:40.528182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:40.528182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:40.528182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:40.528182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:40.528182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:40.528182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:40.528182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:40.531549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 304/610] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:40.533120 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26159fc5-bdeb-459b-99c8-34835382dd82 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:40.533564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26159fc5-bdeb-459b-99c8-34835382dd82 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:40.533993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26159fc5-bdeb-459b-99c8-34835382dd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:40.538449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:40.561294 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-26159fc5-bdeb-459b-99c8-34835382dd82 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:40.562775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-26159fc5-bdeb-459b-99c8-34835382dd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:40.570078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed11731-3975-4c21-947b-257cd324c92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:40.570777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed11731-3975-4c21-947b-257cd324c92e None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:40.571291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed11731-3975-4c21-947b-257cd324c92e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:40.571291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed11731-3975-4c21-947b-257cd324c92e None admin] PATH_INFO: `/v3/projects/425048ffd13448cd8fe815333a9b798c/users/6160b373ace44ccfaf557b557343bf32/roles/fc3db8095f31464ca4c378645d1d078b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:40.591284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed11731-3975-4c21-947b-257cd324c92e None admin] RBAC: Authorizing `identity:create_grant(project_id=425048ffd13448cd8fe815333a9b798c, user_id=6160b373ace44ccfaf557b557343bf32, role_id=fc3db8095f31464ca4c378645d1d078b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:40.593514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed11731-3975-4c21-947b-257cd324c92e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:40.609780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 307/611] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:23:40 2022] PUT /identity/v3/projects/425048ffd13448cd8fe815333a9b798c/users/6160b373ace44ccfaf557b557343bf32/roles/fc3db8095f31464ca4c378645d1d078b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:23:40.611602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed11731-3975-4c21-947b-257cd324c92e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:40.658052 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 305/612] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:40.746080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed11731-3975-4c21-947b-257cd324c92e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:40.798081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5aedd-1812-49b6-a547-b8833941fe73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:40.799204 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5aedd-1812-49b6-a547-b8833941fe73 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:40.799683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5aedd-1812-49b6-a547-b8833941fe73 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:40.800177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5aedd-1812-49b6-a547-b8833941fe73 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:40.801044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5aedd-1812-49b6-a547-b8833941fe73 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:40.801543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5aedd-1812-49b6-a547-b8833941fe73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:40.803107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef5aedd-1812-49b6-a547-b8833941fe73 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:40.817404 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 308/613] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:40.823708 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26159fc5-bdeb-459b-99c8-34835382dd82 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:40.858033 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:40.941033 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26159fc5-bdeb-459b-99c8-34835382dd82 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:40.975568 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403299e-b2e5-4630-b428-74859ef99b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:40.976233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403299e-b2e5-4630-b428-74859ef99b01 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:40.976507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403299e-b2e5-4630-b428-74859ef99b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:40.976776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403299e-b2e5-4630-b428-74859ef99b01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:40.988052 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403299e-b2e5-4630-b428-74859ef99b01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:40.989511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403299e-b2e5-4630-b428-74859ef99b01 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:41.003290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5aedd-1812-49b6-a547-b8833941fe73 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-a403299e-b2e5-4630-b428-74859ef99b01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:41.004176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:41.005077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:41.005953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:41.005953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:41.005953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:41.005953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 306/614] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:41.011345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403299e-b2e5-4630-b428-74859ef99b01 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:41.041060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:41.095832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:41.125192 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403299e-b2e5-4630-b428-74859ef99b01 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:41.163305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dd281-8cb6-4fa7-a550-5391fb1f4e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:41.163963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dd281-8cb6-4fa7-a550-5391fb1f4e12 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:41.164238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dd281-8cb6-4fa7-a550-5391fb1f4e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:41.164500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dd281-8cb6-4fa7-a550-5391fb1f4e12 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:41.165323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dd281-8cb6-4fa7-a550-5391fb1f4e12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:41.165728 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dd281-8cb6-4fa7-a550-5391fb1f4e12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:41.167152 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dd281-8cb6-4fa7-a550-5391fb1f4e12 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:41.184101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 307/615] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:41.189986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dd281-8cb6-4fa7-a550-5391fb1f4e12 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:41.265866 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5aedd-1812-49b6-a547-b8833941fe73 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:41.306249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dd281-8cb6-4fa7-a550-5391fb1f4e12 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:41.341525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:41.344587 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8467a9-676b-43ea-8db0-00bc66c8fb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:41.345421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8467a9-676b-43ea-8db0-00bc66c8fb33 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:41.345693 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8467a9-676b-43ea-8db0-00bc66c8fb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:41.346018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8467a9-676b-43ea-8db0-00bc66c8fb33 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:41.354826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8467a9-676b-43ea-8db0-00bc66c8fb33 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:41.356603 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8467a9-676b-43ea-8db0-00bc66c8fb33 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-0a8467a9-676b-43ea-8db0-00bc66c8fb33 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:41.369296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:41.370369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:41.371334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:41.371334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:41.371334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:41.371334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:41.371334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:41.371334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:41.371334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:41.371334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:41.371334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:23:41.371334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:41.371334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 308/616] 10.222.0.62 () {56 vars in 1093 bytes} [Thu Sep 1 16:23:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:41.374276 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d24282-424b-4c15-94e1-2983962cb8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:41.375591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d24282-424b-4c15-94e1-2983962cb8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:41.376126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d24282-424b-4c15-94e1-2983962cb8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:41.376306 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8467a9-676b-43ea-8db0-00bc66c8fb33 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:41.376760 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d24282-424b-4c15-94e1-2983962cb8f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:41.399214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d24282-424b-4c15-94e1-2983962cb8f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b3d24282-424b-4c15-94e1-2983962cb8f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:41.407167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:41.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:41.408972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:41.408972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:41.408972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:41.408972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:41.408972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:41.408972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:41.408972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:41.408972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:41.408972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:41.408972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:41.408972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:41.408972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:41.414167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 309/617] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:41.418996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d24282-424b-4c15-94e1-2983962cb8f2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:41.564045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8467a9-676b-43ea-8db0-00bc66c8fb33 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:41.572284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:41.600691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d24282-424b-4c15-94e1-2983962cb8f2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:41.632525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f7ef0-14ce-4d0d-a46c-9fd83d9c28cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:41.633143 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f7ef0-14ce-4d0d-a46c-9fd83d9c28cf None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:41.633423 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f7ef0-14ce-4d0d-a46c-9fd83d9c28cf None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:41.633738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f7ef0-14ce-4d0d-a46c-9fd83d9c28cf None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:41.634589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f7ef0-14ce-4d0d-a46c-9fd83d9c28cf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:41.634894 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f7ef0-14ce-4d0d-a46c-9fd83d9c28cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:41.636421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f7ef0-14ce-4d0d-a46c-9fd83d9c28cf None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:41.658948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693403b-440f-40a7-a5a0-4cda77ea1fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:41.659563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693403b-440f-40a7-a5a0-4cda77ea1fba None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:41.659839 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693403b-440f-40a7-a5a0-4cda77ea1fba None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:41.660140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693403b-440f-40a7-a5a0-4cda77ea1fba None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:41.660482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 309/618] 10.222.0.62 () {56 vars in 1133 bytes} [Thu Sep 1 16:23:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:41.660807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693403b-440f-40a7-a5a0-4cda77ea1fba None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:41.661107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693403b-440f-40a7-a5a0-4cda77ea1fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:41.661734 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1923352-87a8-48b5-b7dd-a943178b97e9 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:41.662092 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1923352-87a8-48b5-b7dd-a943178b97e9 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:41.662379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1923352-87a8-48b5-b7dd-a943178b97e9 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:41.662545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7693403b-440f-40a7-a5a0-4cda77ea1fba None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:41.667611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 310/619] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:41.668532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1923352-87a8-48b5-b7dd-a943178b97e9 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:41.672436 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 310/620] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:41.674023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7074c-120e-400c-a422-8bed40cfc2db None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:41.674456 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7074c-120e-400c-a422-8bed40cfc2db None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:41.674869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7074c-120e-400c-a422-8bed40cfc2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:41.696853 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-17f7074c-120e-400c-a422-8bed40cfc2db None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:41.699275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-17f7074c-120e-400c-a422-8bed40cfc2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:41.728148 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:41.795922 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 311/621] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:41.795922 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7074c-120e-400c-a422-8bed40cfc2db None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:41.798026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1923352-87a8-48b5-b7dd-a943178b97e9 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:41.869710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520d797-6b8c-431d-808c-6115c4217521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:41.870651 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520d797-6b8c-431d-808c-6115c4217521 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:41.871102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520d797-6b8c-431d-808c-6115c4217521 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:41.871543 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520d797-6b8c-431d-808c-6115c4217521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:41.891411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:41.931303 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7074c-120e-400c-a422-8bed40cfc2db None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:41.994608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520d797-6b8c-431d-808c-6115c4217521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:41.996565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520d797-6b8c-431d-808c-6115c4217521 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:42.008405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35ea3c-5191-4f2f-ba8d-c455b2cfe921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:42.009053 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35ea3c-5191-4f2f-ba8d-c455b2cfe921 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:42.009886 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35ea3c-5191-4f2f-ba8d-c455b2cfe921 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:42.010170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35ea3c-5191-4f2f-ba8d-c455b2cfe921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:42.031333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35ea3c-5191-4f2f-ba8d-c455b2cfe921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:42.034312 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35ea3c-5191-4f2f-ba8d-c455b2cfe921 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-bd35ea3c-5191-4f2f-ba8d-c455b2cfe921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:42.069911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:42.070981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:42.072232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:42.072232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:42.072232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:42.080043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 312/622] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:42.085002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35ea3c-5191-4f2f-ba8d-c455b2cfe921 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:42.100999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:42.177613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 311/623] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:42.178806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520d797-6b8c-431d-808c-6115c4217521 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:42.223137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35ea3c-5191-4f2f-ba8d-c455b2cfe921 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:42.260546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:42.272214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c8e61-43e5-4bd2-b9f1-7237a7964618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:42.273021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c8e61-43e5-4bd2-b9f1-7237a7964618 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:42.273367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c8e61-43e5-4bd2-b9f1-7237a7964618 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:42.273721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c8e61-43e5-4bd2-b9f1-7237a7964618 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:42.274545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c8e61-43e5-4bd2-b9f1-7237a7964618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:42.275127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c8e61-43e5-4bd2-b9f1-7237a7964618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:42.276833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c8e61-43e5-4bd2-b9f1-7237a7964618 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:42.295853 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 313/624] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:42.297211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c8e61-43e5-4bd2-b9f1-7237a7964618 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:42.326886 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520d797-6b8c-431d-808c-6115c4217521 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:42.365393 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9d4a0-8a33-4380-8896-b3aa89becfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:42.366032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9d4a0-8a33-4380-8896-b3aa89becfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:42.366308 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9d4a0-8a33-4380-8896-b3aa89becfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:42.366577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9d4a0-8a33-4380-8896-b3aa89becfc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:42.377639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9d4a0-8a33-4380-8896-b3aa89becfc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-12b9d4a0-8a33-4380-8896-b3aa89becfc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:42.382551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:42.384781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:42.386934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:42.386934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:42.386934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:42.386934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:42.386934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:42.386934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:42.386934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:42.386934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:42.386934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:42.386934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:42.386934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:42.386934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:42.386934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 312/625] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:42.386934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9d4a0-8a33-4380-8896-b3aa89becfc6 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:42.397710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:42.423568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c8e61-43e5-4bd2-b9f1-7237a7964618 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:42.464293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d4cbb-e1ab-41a1-921e-6c2009dede95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:42.465345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d4cbb-e1ab-41a1-921e-6c2009dede95 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:42.465806 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d4cbb-e1ab-41a1-921e-6c2009dede95 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:42.466248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d4cbb-e1ab-41a1-921e-6c2009dede95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:42.478523 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d4cbb-e1ab-41a1-921e-6c2009dede95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:42.480444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d4cbb-e1ab-41a1-921e-6c2009dede95 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:42.491403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9d4a0-8a33-4380-8896-b3aa89becfc6 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-857d4cbb-e1ab-41a1-921e-6c2009dede95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:42.497535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:42.498477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:42.499351 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:42.499351 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:42.499351 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:42.505992 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 314/626] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:42.507356 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d4cbb-e1ab-41a1-921e-6c2009dede95 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:42.527347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03282a1e-66c8-4860-8cdb-7b7236e8e1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:42.528020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03282a1e-66c8-4860-8cdb-7b7236e8e1fb None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:42.528401 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03282a1e-66c8-4860-8cdb-7b7236e8e1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:42.528663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03282a1e-66c8-4860-8cdb-7b7236e8e1fb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:42.536577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03282a1e-66c8-4860-8cdb-7b7236e8e1fb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:42.538066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03282a1e-66c8-4860-8cdb-7b7236e8e1fb None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-03282a1e-66c8-4860-8cdb-7b7236e8e1fb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:42.551272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:42.552332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:42.553397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:42.553397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:42.553397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:42.553397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:42.553397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:42.553397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:42.553397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:42.553397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:42.553397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:23:42.553397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:42.553397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 313/627] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:23:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:42.554000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03282a1e-66c8-4860-8cdb-7b7236e8e1fb None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:42.559658 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:42.625372 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d4cbb-e1ab-41a1-921e-6c2009dede95 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:42.630873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:42.660860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bba15e-cc65-4a11-8c32-34ca5a12ef47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:42.661471 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bba15e-cc65-4a11-8c32-34ca5a12ef47 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:42.661753 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bba15e-cc65-4a11-8c32-34ca5a12ef47 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:42.662018 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bba15e-cc65-4a11-8c32-34ca5a12ef47 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:42.662731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bba15e-cc65-4a11-8c32-34ca5a12ef47 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:42.662976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bba15e-cc65-4a11-8c32-34ca5a12ef47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:42.664396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bba15e-cc65-4a11-8c32-34ca5a12ef47 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:42.665944 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03282a1e-66c8-4860-8cdb-7b7236e8e1fb None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:42.673147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 315/628] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:42.673863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bba15e-cc65-4a11-8c32-34ca5a12ef47 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:42.709944 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05034a3-cfdb-426b-be8a-e803b84fe566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:42.710977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05034a3-cfdb-426b-be8a-e803b84fe566 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:42.711614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05034a3-cfdb-426b-be8a-e803b84fe566 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:42.712115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05034a3-cfdb-426b-be8a-e803b84fe566 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:42.713158 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05034a3-cfdb-426b-be8a-e803b84fe566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:42.713688 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05034a3-cfdb-426b-be8a-e803b84fe566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:42.715479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05034a3-cfdb-426b-be8a-e803b84fe566 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:42.729618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 314/629] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:42.731296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007ba57-fdf8-47a1-a5d3-0819235d86cc None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:42.731773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007ba57-fdf8-47a1-a5d3-0819235d86cc None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:42.732318 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007ba57-fdf8-47a1-a5d3-0819235d86cc None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:42.734276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 315/630] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:42.735186 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007ba57-fdf8-47a1-a5d3-0819235d86cc None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:42.808825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bba15e-cc65-4a11-8c32-34ca5a12ef47 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:42.810442 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:42.855000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007ba57-fdf8-47a1-a5d3-0819235d86cc None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:42.860032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b175b-10d6-4f17-b7ba-bb0cb10ae04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:42.860783 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b175b-10d6-4f17-b7ba-bb0cb10ae04d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:42.861128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b175b-10d6-4f17-b7ba-bb0cb10ae04d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:42.861462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b175b-10d6-4f17-b7ba-bb0cb10ae04d None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:42.862290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b175b-10d6-4f17-b7ba-bb0cb10ae04d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:42.862657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b175b-10d6-4f17-b7ba-bb0cb10ae04d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:42.864211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b175b-10d6-4f17-b7ba-bb0cb10ae04d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:42.894297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 316/631] 10.222.0.62 () {56 vars in 1135 bytes} [Thu Sep 1 16:23:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:42.895654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b175b-10d6-4f17-b7ba-bb0cb10ae04d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:42.897879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:42.905554 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6329f31-2c49-4f6f-958b-d4d6aa19b741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:42.906234 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6329f31-2c49-4f6f-958b-d4d6aa19b741 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:42.906484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6329f31-2c49-4f6f-958b-d4d6aa19b741 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:42.906744 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6329f31-2c49-4f6f-958b-d4d6aa19b741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:42.918248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6329f31-2c49-4f6f-958b-d4d6aa19b741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:42.919730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6329f31-2c49-4f6f-958b-d4d6aa19b741 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-e6329f31-2c49-4f6f-958b-d4d6aa19b741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:42.937500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:42.938473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:42.939338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:42.939338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:42.939338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:42.939338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 316/632] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:42.941646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020ff7f-eaef-4580-9c56-f496f5d5512a None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:42.941923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020ff7f-eaef-4580-9c56-f496f5d5512a None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:42.942592 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020ff7f-eaef-4580-9c56-f496f5d5512a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:42.963220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-5020ff7f-eaef-4580-9c56-f496f5d5512a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:42.964451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5020ff7f-eaef-4580-9c56-f496f5d5512a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:43.010752 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b175b-10d6-4f17-b7ba-bb0cb10ae04d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:43.047434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 317/633] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:43.048405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020ff7f-eaef-4580-9c56-f496f5d5512a None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:43.096123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b662b3e-5c56-44cd-9ba8-0e875315dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:43.097193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b662b3e-5c56-44cd-9ba8-0e875315dc0b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:43.097469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b662b3e-5c56-44cd-9ba8-0e875315dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:43.097736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b662b3e-5c56-44cd-9ba8-0e875315dc0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:43.107037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b662b3e-5c56-44cd-9ba8-0e875315dc0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:43.108687 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b662b3e-5c56-44cd-9ba8-0e875315dc0b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2b662b3e-5c56-44cd-9ba8-0e875315dc0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:43.120749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:43.121674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:43.122547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:43.122547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:43.122547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:43.122547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:43.122547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:43.122547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:43.122547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:43.122547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:43.122547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 317/634] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:43.123669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be66e4b-d0f5-4e32-b4b3-845ef8565622 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:43.123669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be66e4b-d0f5-4e32-b4b3-845ef8565622 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:43.123669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be66e4b-d0f5-4e32-b4b3-845ef8565622 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:43.125026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 318/635] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:43.125510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be66e4b-d0f5-4e32-b4b3-845ef8565622 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:43.148125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:43.170026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020ff7f-eaef-4580-9c56-f496f5d5512a None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:43.183604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:43.206633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eccc93-b7c6-4fbd-a361-ef06342e8120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:43.207669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eccc93-b7c6-4fbd-a361-ef06342e8120 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:43.207982 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eccc93-b7c6-4fbd-a361-ef06342e8120 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:43.208247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eccc93-b7c6-4fbd-a361-ef06342e8120 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:43.220820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eccc93-b7c6-4fbd-a361-ef06342e8120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:43.222269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eccc93-b7c6-4fbd-a361-ef06342e8120 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:43.232666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be66e4b-d0f5-4e32-b4b3-845ef8565622 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-41eccc93-b7c6-4fbd-a361-ef06342e8120 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:43.236912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:43.237889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:43.238806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:43.238806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:43.238806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:43.243006 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 318/636] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:43.244055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eccc93-b7c6-4fbd-a361-ef06342e8120 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:43.271147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245dd82e-f3be-43fd-98a5-122049335220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:43.271684 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dd82e-f3be-43fd-98a5-122049335220 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:43.271985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dd82e-f3be-43fd-98a5-122049335220 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:43.272252 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dd82e-f3be-43fd-98a5-122049335220 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:43.272895 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245dd82e-f3be-43fd-98a5-122049335220 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:43.273193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245dd82e-f3be-43fd-98a5-122049335220 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:43.274562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245dd82e-f3be-43fd-98a5-122049335220 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:43.290352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 319/637] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:43.290352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2245-816a-45e0-813b-686d9dfc5d8c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:43.290352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2245-816a-45e0-813b-686d9dfc5d8c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:43.290352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2245-816a-45e0-813b-686d9dfc5d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:43.308559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c69e2245-816a-45e0-813b-686d9dfc5d8c None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:43.309841 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c69e2245-816a-45e0-813b-686d9dfc5d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:43.350860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eccc93-b7c6-4fbd-a361-ef06342e8120 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:43.387265 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5b700-e166-4ea6-ac50-7b81d010460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:43.387870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5b700-e166-4ea6-ac50-7b81d010460b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:43.388165 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5b700-e166-4ea6-ac50-7b81d010460b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:43.388441 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5b700-e166-4ea6-ac50-7b81d010460b None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:43.389066 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 320/638] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:43.389251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5b700-e166-4ea6-ac50-7b81d010460b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:43.389408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5b700-e166-4ea6-ac50-7b81d010460b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:43.389896 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2245-816a-45e0-813b-686d9dfc5d8c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:43.390838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5b700-e166-4ea6-ac50-7b81d010460b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:43.401581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:43.408861 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 319/639] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:43.409937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5b700-e166-4ea6-ac50-7b81d010460b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:43.435832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:43.505819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2245-816a-45e0-813b-686d9dfc5d8c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:43.512505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5b700-e166-4ea6-ac50-7b81d010460b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:43.541996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c70b88-904a-47b8-9d8d-90b0225b7a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:43.542509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c70b88-904a-47b8-9d8d-90b0225b7a79 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:43.542781 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c70b88-904a-47b8-9d8d-90b0225b7a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:43.543045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c70b88-904a-47b8-9d8d-90b0225b7a79 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:43.543672 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c70b88-904a-47b8-9d8d-90b0225b7a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:43.544006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c70b88-904a-47b8-9d8d-90b0225b7a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:43.545405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c70b88-904a-47b8-9d8d-90b0225b7a79 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:43.546496 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f78cb6-566b-461d-8114-e1281953431d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:43.547130 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f78cb6-566b-461d-8114-e1281953431d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:43.547407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f78cb6-566b-461d-8114-e1281953431d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:43.547682 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f78cb6-566b-461d-8114-e1281953431d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:43.554803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f78cb6-566b-461d-8114-e1281953431d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:43.556330 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f78cb6-566b-461d-8114-e1281953431d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:43.561911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 321/640] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:43.561911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c70b88-904a-47b8-9d8d-90b0225b7a79 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-17f78cb6-566b-461d-8114-e1281953431d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:43.569993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:43.571018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:43.578207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:43.578207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:43.578207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:43.578207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:43.578207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:43.578207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:43.578207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:43.578207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:43.578207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:23:43.578207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:43.578207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 320/641] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:43.578207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f78cb6-566b-461d-8114-e1281953431d None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:43.638938 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:43.666193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:43.671762 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c70b88-904a-47b8-9d8d-90b0225b7a79 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:43.682055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f78cb6-566b-461d-8114-e1281953431d None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:43.703741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56f833-922d-43b3-87b1-e6cb03f5270c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:43.704397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56f833-922d-43b3-87b1-e6cb03f5270c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:43.704669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56f833-922d-43b3-87b1-e6cb03f5270c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:43.704935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56f833-922d-43b3-87b1-e6cb03f5270c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:43.716117 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56f833-922d-43b3-87b1-e6cb03f5270c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:43.717510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c65c99-1165-4337-a386-57a6929093e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:43.718114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c65c99-1165-4337-a386-57a6929093e3 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:43.718387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c65c99-1165-4337-a386-57a6929093e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:43.718654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c65c99-1165-4337-a386-57a6929093e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:43.719368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c65c99-1165-4337-a386-57a6929093e3 None admin] RBAC: Adding query filter params (role.id=fc3db8095f31464ca4c378645d1d078b, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=2912c02ff6db4cbc97c80cd923238136) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:43.719670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c65c99-1165-4337-a386-57a6929093e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:43.721157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c65c99-1165-4337-a386-57a6929093e3 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ea56f833-922d-43b3-87b1-e6cb03f5270c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:43.721413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:43.722494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:43.723755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:43.723755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:43.723755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:43.723755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:43.723755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:43.723755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:43.723755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:43.723755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:43.723755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:43.723755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:43.723755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:43.723755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:43.723755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 322/642] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:43.723755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56f833-922d-43b3-87b1-e6cb03f5270c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:43.738615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 321/643] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:43 2022] GET /identity/v3/role_assignments?user.id=2912c02ff6db4cbc97c80cd923238136&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=fc3db8095f31464ca4c378645d1d078b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:43.738615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c65c99-1165-4337-a386-57a6929093e3 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:43.850293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56f833-922d-43b3-87b1-e6cb03f5270c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:43.854722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c65c99-1165-4337-a386-57a6929093e3 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:43.884420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:43.892456 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f47b18-bbbb-4b43-95ab-de8f59f96a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:43.893560 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f47b18-bbbb-4b43-95ab-de8f59f96a3c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:43.893779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a3206-ebd1-48af-9845-9e504ec6dd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:43.893892 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f47b18-bbbb-4b43-95ab-de8f59f96a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:43.894099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f47b18-bbbb-4b43-95ab-de8f59f96a3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:43.894447 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a3206-ebd1-48af-9845-9e504ec6dd15 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:43.894724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a3206-ebd1-48af-9845-9e504ec6dd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:43.894995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a3206-ebd1-48af-9845-9e504ec6dd15 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:43.895765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a3206-ebd1-48af-9845-9e504ec6dd15 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:43.896087 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a3206-ebd1-48af-9845-9e504ec6dd15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:43.897901 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a3206-ebd1-48af-9845-9e504ec6dd15 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:43.902292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f47b18-bbbb-4b43-95ab-de8f59f96a3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:43.903718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f47b18-bbbb-4b43-95ab-de8f59f96a3c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-25f47b18-bbbb-4b43-95ab-de8f59f96a3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:43.915939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:43.916846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:43.917883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:43.917883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:43.917883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:43.917883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:43.917883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:43.917883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:43.917883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:43.917883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:43.918320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 322/644] 10.222.0.62 () {56 vars in 1139 bytes} [Thu Sep 1 16:23:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:43.918853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a3206-ebd1-48af-9845-9e504ec6dd15 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:43.922562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 323/645] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:43.923439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f47b18-bbbb-4b43-95ab-de8f59f96a3c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:43.928739 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:44.028165 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a3206-ebd1-48af-9845-9e504ec6dd15 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:44.038477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f47b18-bbbb-4b43-95ab-de8f59f96a3c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:44.065719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e6e89-0222-4050-95dc-23615ff6c121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:44.066351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e6e89-0222-4050-95dc-23615ff6c121 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:44.066626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e6e89-0222-4050-95dc-23615ff6c121 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:44.066888 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e6e89-0222-4050-95dc-23615ff6c121 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:44.075764 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8e423-8541-47f4-aa9a-26b0d90d7898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:44.076596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8e423-8541-47f4-aa9a-26b0d90d7898 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:44.077015 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8e423-8541-47f4-aa9a-26b0d90d7898 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:44.077424 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8e423-8541-47f4-aa9a-26b0d90d7898 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:44.077741 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e6e89-0222-4050-95dc-23615ff6c121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:44.078381 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8e423-8541-47f4-aa9a-26b0d90d7898 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:44.078845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8e423-8541-47f4-aa9a-26b0d90d7898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:44.080383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8e423-8541-47f4-aa9a-26b0d90d7898 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-5f7e6e89-0222-4050-95dc-23615ff6c121 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:44.082817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:44.083921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:44.084767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:44.084767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:44.084767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:44.084767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:44.084767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:44.084767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:44.084767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:44.084767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:44.084767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:44.084767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:44.084767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:44.084767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:44.084767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 323/646] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:44.085449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e6e89-0222-4050-95dc-23615ff6c121 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:44.096824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 324/647] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:44.097921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8e423-8541-47f4-aa9a-26b0d90d7898 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:44.153916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:44.157951 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:44.195240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e6e89-0222-4050-95dc-23615ff6c121 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:44.210111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8e423-8541-47f4-aa9a-26b0d90d7898 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:44.231724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f57ec-ad79-49a9-897a-b340d7ae4014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:44.232366 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f57ec-ad79-49a9-897a-b340d7ae4014 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:44.232640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f57ec-ad79-49a9-897a-b340d7ae4014 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:44.232907 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f57ec-ad79-49a9-897a-b340d7ae4014 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:44.252771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67491d3a-4399-4af8-8a69-da4b16622c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:44.253712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67491d3a-4399-4af8-8a69-da4b16622c37 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:44.254169 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67491d3a-4399-4af8-8a69-da4b16622c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:44.254657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67491d3a-4399-4af8-8a69-da4b16622c37 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:44.255324 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67491d3a-4399-4af8-8a69-da4b16622c37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:44.255620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67491d3a-4399-4af8-8a69-da4b16622c37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:44.257093 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67491d3a-4399-4af8-8a69-da4b16622c37 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:44.275036 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 325/648] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:44.275036 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67491d3a-4399-4af8-8a69-da4b16622c37 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:44.318033 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f57ec-ad79-49a9-897a-b340d7ae4014 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:44.319500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f57ec-ad79-49a9-897a-b340d7ae4014 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:44.377210 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67491d3a-4399-4af8-8a69-da4b16622c37 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:44.390917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:44.393038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:44.418126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c330b31-3554-446b-ad0f-2d0cfd48a1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:44.418777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c330b31-3554-446b-ad0f-2d0cfd48a1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:44.419054 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c330b31-3554-446b-ad0f-2d0cfd48a1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:44.419431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c330b31-3554-446b-ad0f-2d0cfd48a1c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:44.420121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c330b31-3554-446b-ad0f-2d0cfd48a1c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:44.420423 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c330b31-3554-446b-ad0f-2d0cfd48a1c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:44.421906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c330b31-3554-446b-ad0f-2d0cfd48a1c5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:44.428668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 324/649] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:44.429574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f57ec-ad79-49a9-897a-b340d7ae4014 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:44.439589 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 326/650] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:44.439589 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c330b31-3554-446b-ad0f-2d0cfd48a1c5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:44.540961 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c330b31-3554-446b-ad0f-2d0cfd48a1c5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:44.546911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f57ec-ad79-49a9-897a-b340d7ae4014 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:44.575141 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ede77-07f3-41b1-9ab6-4389ff85ae05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:44.575806 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ede77-07f3-41b1-9ab6-4389ff85ae05 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:44.576125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ede77-07f3-41b1-9ab6-4389ff85ae05 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:44.576394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ede77-07f3-41b1-9ab6-4389ff85ae05 None admin] PATH_INFO: `/v3/projects/425048ffd13448cd8fe815333a9b798c/users/56f9bc7262b64bbab0f670576c6bdfdd/roles/fc3db8095f31464ca4c378645d1d078b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:44.584494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18976c63-92b5-4f0e-9a80-d44b70bcf916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:44.585452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18976c63-92b5-4f0e-9a80-d44b70bcf916 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:44.585801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18976c63-92b5-4f0e-9a80-d44b70bcf916 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:44.586068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18976c63-92b5-4f0e-9a80-d44b70bcf916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:44.592259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ede77-07f3-41b1-9ab6-4389ff85ae05 None admin] RBAC: Authorizing `identity:create_grant(project_id=425048ffd13448cd8fe815333a9b798c, user_id=56f9bc7262b64bbab0f670576c6bdfdd, role_id=fc3db8095f31464ca4c378645d1d078b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:44.593719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ede77-07f3-41b1-9ab6-4389ff85ae05 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:44.598213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18976c63-92b5-4f0e-9a80-d44b70bcf916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:44.599737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18976c63-92b5-4f0e-9a80-d44b70bcf916 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:44.610408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 327/651] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:23:44 2022] PUT /identity/v3/projects/425048ffd13448cd8fe815333a9b798c/users/56f9bc7262b64bbab0f670576c6bdfdd/roles/fc3db8095f31464ca4c378645d1d078b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:23:44.611042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ede77-07f3-41b1-9ab6-4389ff85ae05 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-18976c63-92b5-4f0e-9a80-d44b70bcf916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:44.617463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:44.618550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:44.620388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:44.620388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:44.620388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:44.620388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 325/652] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:44.620643 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18976c63-92b5-4f0e-9a80-d44b70bcf916 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:44.642476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:44.651680 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:44.743673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18976c63-92b5-4f0e-9a80-d44b70bcf916 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:44.758817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ede77-07f3-41b1-9ab6-4389ff85ae05 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:44.786809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee24f0-7650-4c7a-a591-9243c76e6000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:44.787579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee24f0-7650-4c7a-a591-9243c76e6000 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:44.787941 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee24f0-7650-4c7a-a591-9243c76e6000 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:44.788282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee24f0-7650-4c7a-a591-9243c76e6000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:44.805855 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee24f0-7650-4c7a-a591-9243c76e6000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:44.806269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee24f0-7650-4c7a-a591-9243c76e6000 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:44.811842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae41d3-2ea9-4181-a33f-dc6aefbb7b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:44.812532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae41d3-2ea9-4181-a33f-dc6aefbb7b2a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:44.812813 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae41d3-2ea9-4181-a33f-dc6aefbb7b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:44.813082 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae41d3-2ea9-4181-a33f-dc6aefbb7b2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-62ee24f0-7650-4c7a-a591-9243c76e6000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:44.825765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:44.826801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:44.827678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:44.827678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:44.827678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:44.827678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 326/653] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:44.827678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee24f0-7650-4c7a-a591-9243c76e6000 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:44.833332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae41d3-2ea9-4181-a33f-dc6aefbb7b2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:44.833887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae41d3-2ea9-4181-a33f-dc6aefbb7b2a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-0bae41d3-2ea9-4181-a33f-dc6aefbb7b2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:44.849605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:44.850647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:44.851532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:44.851532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:44.851532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:44.851532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 328/654] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:44.851532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae41d3-2ea9-4181-a33f-dc6aefbb7b2a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:44.925829 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:44.936153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:44.965883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee24f0-7650-4c7a-a591-9243c76e6000 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:44.971150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae41d3-2ea9-4181-a33f-dc6aefbb7b2a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:45.002385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045e528-2c99-4b41-b88c-9817a948d36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:45.002930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045e528-2c99-4b41-b88c-9817a948d36d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:45.003205 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045e528-2c99-4b41-b88c-9817a948d36d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:45.003535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045e528-2c99-4b41-b88c-9817a948d36d None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:45.004224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045e528-2c99-4b41-b88c-9817a948d36d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:45.004761 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045e528-2c99-4b41-b88c-9817a948d36d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:45.006493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045e528-2c99-4b41-b88c-9817a948d36d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:45.018733 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86d3d1-4bb8-4a38-98a0-bf9547038dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:45.019828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86d3d1-4bb8-4a38-98a0-bf9547038dad None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:45.020347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86d3d1-4bb8-4a38-98a0-bf9547038dad None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:45.020855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86d3d1-4bb8-4a38-98a0-bf9547038dad None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:45.021696 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86d3d1-4bb8-4a38-98a0-bf9547038dad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:45.022196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86d3d1-4bb8-4a38-98a0-bf9547038dad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:45.023827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86d3d1-4bb8-4a38-98a0-bf9547038dad None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:45.025242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 327/655] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:45.026288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045e528-2c99-4b41-b88c-9817a948d36d None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:45.044932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 329/656] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:45.044932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86d3d1-4bb8-4a38-98a0-bf9547038dad None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:45.174728 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045e528-2c99-4b41-b88c-9817a948d36d None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:45.178322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86d3d1-4bb8-4a38-98a0-bf9547038dad None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:45.218623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:45.231083 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef529f6c-0c72-41c2-b80e-47c4356973b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:45.231704 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef529f6c-0c72-41c2-b80e-47c4356973b2 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:45.232012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef529f6c-0c72-41c2-b80e-47c4356973b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:45.232285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef529f6c-0c72-41c2-b80e-47c4356973b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:45.232934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef529f6c-0c72-41c2-b80e-47c4356973b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:45.233235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef529f6c-0c72-41c2-b80e-47c4356973b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:45.234353 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be8e54-b7bb-4ef1-85af-743652de5581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:45.234624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef529f6c-0c72-41c2-b80e-47c4356973b2 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:45.235423 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be8e54-b7bb-4ef1-85af-743652de5581 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:45.235938 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be8e54-b7bb-4ef1-85af-743652de5581 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:45.236659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be8e54-b7bb-4ef1-85af-743652de5581 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:45.240508 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:45.248718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be8e54-b7bb-4ef1-85af-743652de5581 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:45.250519 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be8e54-b7bb-4ef1-85af-743652de5581 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:45.254443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 328/657] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:45.254443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef529f6c-0c72-41c2-b80e-47c4356973b2 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a6be8e54-b7bb-4ef1-85af-743652de5581 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:45.266329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:45.267408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:45.268461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:45.268461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:45.268461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:45.268461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:45.268461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:45.268461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:45.268461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:45.268461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:45.268461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:23:45.268461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:45.269943 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 330/658] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:45.270234 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be8e54-b7bb-4ef1-85af-743652de5581 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:45.372208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef529f6c-0c72-41c2-b80e-47c4356973b2 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:45.410530 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60068456-3810-4a0b-b433-2eea12583867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:45.411630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60068456-3810-4a0b-b433-2eea12583867 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:45.411852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60068456-3810-4a0b-b433-2eea12583867 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:45.412178 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60068456-3810-4a0b-b433-2eea12583867 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:45.414845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be8e54-b7bb-4ef1-85af-743652de5581 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:45.420678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60068456-3810-4a0b-b433-2eea12583867 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:45.422288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60068456-3810-4a0b-b433-2eea12583867 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-60068456-3810-4a0b-b433-2eea12583867 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:45.434525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:45.435701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:45.436887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:45.436887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:45.436887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:45.436887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:45.436887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:45.436887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:45.436887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:45.436887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:45.436887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:23:45.436887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:45.443053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 329/659] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:45.444071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60068456-3810-4a0b-b433-2eea12583867 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:45.460852 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d864e0-dbbe-40e2-84bd-f5bbed2dcad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:45.462244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d864e0-dbbe-40e2-84bd-f5bbed2dcad3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:45.462774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d864e0-dbbe-40e2-84bd-f5bbed2dcad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:45.463521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d864e0-dbbe-40e2-84bd-f5bbed2dcad3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:45.472904 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d864e0-dbbe-40e2-84bd-f5bbed2dcad3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:45.474885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d864e0-dbbe-40e2-84bd-f5bbed2dcad3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-64d864e0-dbbe-40e2-84bd-f5bbed2dcad3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:45.490073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:45.491151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:45.492263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:45.492263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:45.492263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:45.492263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:45.492263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:45.492263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:45.492263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:45.492263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:45.494892 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 331/660] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:45.495987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d864e0-dbbe-40e2-84bd-f5bbed2dcad3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:45.508248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:45.558051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:45.578179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60068456-3810-4a0b-b433-2eea12583867 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:45.605926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d864e0-dbbe-40e2-84bd-f5bbed2dcad3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:45.624227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde72b32-1e40-4157-8dc2-7715a594d887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:45.625418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde72b32-1e40-4157-8dc2-7715a594d887 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:45.625844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde72b32-1e40-4157-8dc2-7715a594d887 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:45.626257 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde72b32-1e40-4157-8dc2-7715a594d887 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:45.627140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde72b32-1e40-4157-8dc2-7715a594d887 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:45.627586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde72b32-1e40-4157-8dc2-7715a594d887 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:45.629442 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde72b32-1e40-4157-8dc2-7715a594d887 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:45.646791 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c77f4-fc3a-4659-8f9d-ae8594dc9cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:45.647458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c77f4-fc3a-4659-8f9d-ae8594dc9cba None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:45.647796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c77f4-fc3a-4659-8f9d-ae8594dc9cba None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:45.648107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c77f4-fc3a-4659-8f9d-ae8594dc9cba None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:45.648853 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c77f4-fc3a-4659-8f9d-ae8594dc9cba None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:45.649157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c77f4-fc3a-4659-8f9d-ae8594dc9cba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:45.650739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891c77f4-fc3a-4659-8f9d-ae8594dc9cba None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:45.651504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 330/661] 10.222.0.62 () {56 vars in 1137 bytes} [Thu Sep 1 16:23:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:45.652962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde72b32-1e40-4157-8dc2-7715a594d887 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:45.677975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 332/662] 10.222.0.62 () {56 vars in 1137 bytes} [Thu Sep 1 16:23:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:45.678912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c77f4-fc3a-4659-8f9d-ae8594dc9cba None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:45.759490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde72b32-1e40-4157-8dc2-7715a594d887 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:45.770635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:45.797386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74017ca-6e72-4db7-a65c-f33d5f3adfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:45.798019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74017ca-6e72-4db7-a65c-f33d5f3adfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:45.798297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74017ca-6e72-4db7-a65c-f33d5f3adfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:45.798567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74017ca-6e72-4db7-a65c-f33d5f3adfe2 None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:45.799269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74017ca-6e72-4db7-a65c-f33d5f3adfe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:45.799574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74017ca-6e72-4db7-a65c-f33d5f3adfe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:45.801102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74017ca-6e72-4db7-a65c-f33d5f3adfe2 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:45.809676 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c77f4-fc3a-4659-8f9d-ae8594dc9cba None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:45.815179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 331/663] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:45.815855 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74017ca-6e72-4db7-a65c-f33d5f3adfe2 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:45.829818 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:45.854099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d0c5d-ba7f-4a64-8eba-78df066d3fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:45.854709 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d0c5d-ba7f-4a64-8eba-78df066d3fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:45.854986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d0c5d-ba7f-4a64-8eba-78df066d3fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:45.855256 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d0c5d-ba7f-4a64-8eba-78df066d3fb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:45.926958 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74017ca-6e72-4db7-a65c-f33d5f3adfe2 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:45.927560 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d0c5d-ba7f-4a64-8eba-78df066d3fb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:45.929107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d0c5d-ba7f-4a64-8eba-78df066d3fb0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:45.962108 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12376b3-78a4-42d8-be85-604588d4d07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:45.963014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12376b3-78a4-42d8-be85-604588d4d07b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:45.963333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12376b3-78a4-42d8-be85-604588d4d07b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:45.963606 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12376b3-78a4-42d8-be85-604588d4d07b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:46.000907 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:46.029829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 333/664] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:46.031189 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f0538-6832-4d39-8a54-d808ab6eab78 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:46.031479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f0538-6832-4d39-8a54-d808ab6eab78 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:46.031751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f0538-6832-4d39-8a54-d808ab6eab78 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:46.037489 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 334/665] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:46.038354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f0538-6832-4d39-8a54-d808ab6eab78 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:46.045912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12376b3-78a4-42d8-be85-604588d4d07b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:46.047722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12376b3-78a4-42d8-be85-604588d4d07b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:46.055607 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:46.145219 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f0538-6832-4d39-8a54-d808ab6eab78 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:46.170720 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 332/666] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:46.170720 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12376b3-78a4-42d8-be85-604588d4d07b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:46.188057 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff21f09-e229-440c-8750-2b2c63d5c9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:46.188670 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21f09-e229-440c-8750-2b2c63d5c9bb None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:46.188936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21f09-e229-440c-8750-2b2c63d5c9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:46.189367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff21f09-e229-440c-8750-2b2c63d5c9bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:46.189909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff21f09-e229-440c-8750-2b2c63d5c9bb None admin] RBAC: Adding query filter params (role.id=fc3db8095f31464ca4c378645d1d078b, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=0718a95189434a5ea901c40bc868bea4) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:46.190208 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff21f09-e229-440c-8750-2b2c63d5c9bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:46.191619 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff21f09-e229-440c-8750-2b2c63d5c9bb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:46.207929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 335/667] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:46 2022] GET /identity/v3/role_assignments?user.id=0718a95189434a5ea901c40bc868bea4&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=fc3db8095f31464ca4c378645d1d078b => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:46.207929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6c75e-fc2c-429d-8e68-c3ddcac67026 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:46.207929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6c75e-fc2c-429d-8e68-c3ddcac67026 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:46.207929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6c75e-fc2c-429d-8e68-c3ddcac67026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:46.226523 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d1e6c75e-fc2c-429d-8e68-c3ddcac67026 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:46.228041 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d1e6c75e-fc2c-429d-8e68-c3ddcac67026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:46.272189 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:46.317650 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 336/668] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:46.319297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6c75e-fc2c-429d-8e68-c3ddcac67026 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:46.321196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12376b3-78a4-42d8-be85-604588d4d07b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:46.331686 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:46.372967 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caf4c5-0930-492f-b01c-7873520f125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:46.374491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caf4c5-0930-492f-b01c-7873520f125e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:46.374936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caf4c5-0930-492f-b01c-7873520f125e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:46.375754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caf4c5-0930-492f-b01c-7873520f125e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:46.390987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96caf4c5-0930-492f-b01c-7873520f125e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:46.393811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96caf4c5-0930-492f-b01c-7873520f125e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-96caf4c5-0930-492f-b01c-7873520f125e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:46.416113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:46.417776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:46.418879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:46.418879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:46.418879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:46.421306 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 333/669] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:46.427399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caf4c5-0930-492f-b01c-7873520f125e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:46.449496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6c75e-fc2c-429d-8e68-c3ddcac67026 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:46.498756 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dffa59-095b-4d82-bd52-edb71731700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:46.499418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dffa59-095b-4d82-bd52-edb71731700f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:46.499722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dffa59-095b-4d82-bd52-edb71731700f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:46.499995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dffa59-095b-4d82-bd52-edb71731700f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:46.514903 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dffa59-095b-4d82-bd52-edb71731700f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:46.517653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dffa59-095b-4d82-bd52-edb71731700f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-d5dffa59-095b-4d82-bd52-edb71731700f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:46.538148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:46.539147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:46.540078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:46.540078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:46.540078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:46.540078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 337/670] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:46.545344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dffa59-095b-4d82-bd52-edb71731700f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:46.564881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caf4c5-0930-492f-b01c-7873520f125e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:46.586761 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:46.602792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e4492-f77d-4114-8a9f-fcaaa4294106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:46.603821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e4492-f77d-4114-8a9f-fcaaa4294106 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:46.604399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e4492-f77d-4114-8a9f-fcaaa4294106 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:46.604875 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e4492-f77d-4114-8a9f-fcaaa4294106 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:46.605709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e4492-f77d-4114-8a9f-fcaaa4294106 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:46.606205 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e4492-f77d-4114-8a9f-fcaaa4294106 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:46.607829 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e4492-f77d-4114-8a9f-fcaaa4294106 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:46.612838 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:46.621649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 334/671] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:46.627655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e4492-f77d-4114-8a9f-fcaaa4294106 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:46.662009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dffa59-095b-4d82-bd52-edb71731700f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:46.699590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12096322-ef25-4f44-8ed1-b5f39e3aaf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:46.700251 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12096322-ef25-4f44-8ed1-b5f39e3aaf2b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:46.700532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12096322-ef25-4f44-8ed1-b5f39e3aaf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:46.700804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12096322-ef25-4f44-8ed1-b5f39e3aaf2b None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:46.701447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12096322-ef25-4f44-8ed1-b5f39e3aaf2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:46.701748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12096322-ef25-4f44-8ed1-b5f39e3aaf2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:46.703123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12096322-ef25-4f44-8ed1-b5f39e3aaf2b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:46.715360 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 338/672] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:46.716104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12096322-ef25-4f44-8ed1-b5f39e3aaf2b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:46.743936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e4492-f77d-4114-8a9f-fcaaa4294106 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:46.783219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de09dd7-a819-4c9e-be6c-dadabb520fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:46.783849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de09dd7-a819-4c9e-be6c-dadabb520fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:46.784156 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de09dd7-a819-4c9e-be6c-dadabb520fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:46.784422 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de09dd7-a819-4c9e-be6c-dadabb520fb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:46.791302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de09dd7-a819-4c9e-be6c-dadabb520fb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:46.792775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de09dd7-a819-4c9e-be6c-dadabb520fb4 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-8de09dd7-a819-4c9e-be6c-dadabb520fb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:46.804316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:46.805271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:46.806266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:46.806266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:46.806266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:46.806266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:46.806266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:46.806266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:46.806266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:46.806266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:46.810280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 335/673] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:46.811636 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e472c71-ff0c-44fb-88bd-357e507ff44a None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:46.811920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e472c71-ff0c-44fb-88bd-357e507ff44a None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:46.812218 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e472c71-ff0c-44fb-88bd-357e507ff44a None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:46.813646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 336/674] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:46.814174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e472c71-ff0c-44fb-88bd-357e507ff44a None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:46.832642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12096322-ef25-4f44-8ed1-b5f39e3aaf2b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:46.854179 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:46.865279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:46.870726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0c31e-db53-4e3a-adf6-f62ae7aa1408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:46.871636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0c31e-db53-4e3a-adf6-f62ae7aa1408 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:46.871636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0c31e-db53-4e3a-adf6-f62ae7aa1408 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:46.871973 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0c31e-db53-4e3a-adf6-f62ae7aa1408 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:46.882387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0c31e-db53-4e3a-adf6-f62ae7aa1408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-12f0c31e-db53-4e3a-adf6-f62ae7aa1408 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:46.887648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:46.888719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:46.889608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:46.889608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:46.889608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:46.889608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:46.889608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:46.889608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:46.889608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:46.889608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:46.889608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:46.889608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:46.889608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:46.889608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:46.889608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 339/675] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:23:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:46.889608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0c31e-db53-4e3a-adf6-f62ae7aa1408 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:46.933663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e472c71-ff0c-44fb-88bd-357e507ff44a None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:46.969532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef57b6-4189-4a76-a9d2-50e6f1e1830c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:46.970171 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef57b6-4189-4a76-a9d2-50e6f1e1830c None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:46.970446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef57b6-4189-4a76-a9d2-50e6f1e1830c None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:46.970710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef57b6-4189-4a76-a9d2-50e6f1e1830c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:46.978042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef57b6-4189-4a76-a9d2-50e6f1e1830c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:46.979589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef57b6-4189-4a76-a9d2-50e6f1e1830c None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-9aef57b6-4189-4a76-a9d2-50e6f1e1830c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:46.991313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:46.992319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:46.993244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:46.993244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:46.993244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:46.993244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:46.993244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:46.993244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:46.993244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:46.993244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:46.993244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 337/676] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:46.994540 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2dc49-3605-4bf8-b3a7-e711180f3dbc None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:46.994540 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2dc49-3605-4bf8-b3a7-e711180f3dbc None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:46.994540 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2dc49-3605-4bf8-b3a7-e711180f3dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:47.001470 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0c31e-db53-4e3a-adf6-f62ae7aa1408 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:47.015933 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-36e2dc49-3605-4bf8-b3a7-e711180f3dbc None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:47.017260 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-36e2dc49-3605-4bf8-b3a7-e711180f3dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:47.040135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33035b-8380-439c-9b1b-95397044e453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:47.041041 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33035b-8380-439c-9b1b-95397044e453 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:47.041406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33035b-8380-439c-9b1b-95397044e453 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:47.041820 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33035b-8380-439c-9b1b-95397044e453 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:47.042846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33035b-8380-439c-9b1b-95397044e453 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:47.043149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33035b-8380-439c-9b1b-95397044e453 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:47.044761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33035b-8380-439c-9b1b-95397044e453 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:47.057486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 340/677] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:47.058318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33035b-8380-439c-9b1b-95397044e453 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:47.094291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 338/678] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:47.095152 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2dc49-3605-4bf8-b3a7-e711180f3dbc None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:47.126793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:47.128912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:47.212596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33035b-8380-439c-9b1b-95397044e453 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:47.223126 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2dc49-3605-4bf8-b3a7-e711180f3dbc None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:47.248508 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcf88e-4803-4515-81e1-533b4d6185d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:47.249225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcf88e-4803-4515-81e1-533b4d6185d8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:47.249626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcf88e-4803-4515-81e1-533b4d6185d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:47.249985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcf88e-4803-4515-81e1-533b4d6185d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:47.250722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcf88e-4803-4515-81e1-533b4d6185d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:47.251124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcf88e-4803-4515-81e1-533b4d6185d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:47.252678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcf88e-4803-4515-81e1-533b4d6185d8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:47.266626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 341/679] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:47.267638 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcf88e-4803-4515-81e1-533b4d6185d8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:47.269740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aea449-47d6-43c5-b4ed-cd3a290d8d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:47.270866 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aea449-47d6-43c5-b4ed-cd3a290d8d2e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:47.272162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aea449-47d6-43c5-b4ed-cd3a290d8d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:47.272449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aea449-47d6-43c5-b4ed-cd3a290d8d2e None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:47.273190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aea449-47d6-43c5-b4ed-cd3a290d8d2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:47.273494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aea449-47d6-43c5-b4ed-cd3a290d8d2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:47.275327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aea449-47d6-43c5-b4ed-cd3a290d8d2e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:47.293465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 339/680] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:47.294357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aea449-47d6-43c5-b4ed-cd3a290d8d2e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:47.389324 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:47.396888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcf88e-4803-4515-81e1-533b4d6185d8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:47.398573 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:47.401817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aea449-47d6-43c5-b4ed-cd3a290d8d2e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:47.433981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7a20e-eaa0-4f18-b602-37c88e9f8a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:47.434615 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7a20e-eaa0-4f18-b602-37c88e9f8a32 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:47.434889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7a20e-eaa0-4f18-b602-37c88e9f8a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:47.435157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7a20e-eaa0-4f18-b602-37c88e9f8a32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:47.435870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7a20e-eaa0-4f18-b602-37c88e9f8a32 None admin] RBAC: Adding query filter params (role.id=fc3db8095f31464ca4c378645d1d078b, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=3cfe36a14a11493397e4018b773d3cc4) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:47.436202 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7a20e-eaa0-4f18-b602-37c88e9f8a32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:47.437644 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7a20e-eaa0-4f18-b602-37c88e9f8a32 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:47.442700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d3b15-6178-41d5-b6fc-15dbde1fb9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:47.443456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d3b15-6178-41d5-b6fc-15dbde1fb9f8 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:47.443757 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 342/681] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:47 2022] GET /identity/v3/role_assignments?user.id=3cfe36a14a11493397e4018b773d3cc4&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=fc3db8095f31464ca4c378645d1d078b => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:47.444208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d3b15-6178-41d5-b6fc-15dbde1fb9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:47.444577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d3b15-6178-41d5-b6fc-15dbde1fb9f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:47.453846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7a20e-eaa0-4f18-b602-37c88e9f8a32 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:47.458982 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d3b15-6178-41d5-b6fc-15dbde1fb9f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:47.461309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d3b15-6178-41d5-b6fc-15dbde1fb9f8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-355d3b15-6178-41d5-b6fc-15dbde1fb9f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:47.483718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:47.484908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:47.486642 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:47.486642 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:47.486642 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:47.490676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 340/682] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:47.492518 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d3b15-6178-41d5-b6fc-15dbde1fb9f8 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:47.600082 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7a20e-eaa0-4f18-b602-37c88e9f8a32 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:47.643024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c8c8f-0486-414f-9c71-ff2525d6900d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:47.644864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c8c8f-0486-414f-9c71-ff2525d6900d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:47.644864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c8c8f-0486-414f-9c71-ff2525d6900d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:47.644864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c8c8f-0486-414f-9c71-ff2525d6900d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:47.645614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c8c8f-0486-414f-9c71-ff2525d6900d None admin] RBAC: Adding query filter params (role.id=fc3db8095f31464ca4c378645d1d078b, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=6160b373ace44ccfaf557b557343bf32) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:47.645614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c8c8f-0486-414f-9c71-ff2525d6900d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:47.647414 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c8c8f-0486-414f-9c71-ff2525d6900d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:47.652296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d3b15-6178-41d5-b6fc-15dbde1fb9f8 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:47.659052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 343/683] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:47 2022] GET /identity/v3/role_assignments?user.id=6160b373ace44ccfaf557b557343bf32&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=fc3db8095f31464ca4c378645d1d078b => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:47.659052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c8c8f-0486-414f-9c71-ff2525d6900d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:47.709738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e9600-2c70-4cdc-9115-b0978a78e883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:47.710965 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9600-2c70-4cdc-9115-b0978a78e883 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:47.711529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9600-2c70-4cdc-9115-b0978a78e883 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:47.712870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9600-2c70-4cdc-9115-b0978a78e883 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:47.718860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:47.730992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e9600-2c70-4cdc-9115-b0978a78e883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-0f9e9600-2c70-4cdc-9115-b0978a78e883 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:47.739707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:47.740986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:47.742039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:47.742039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:47.742039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:47.742039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:47.742039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:47.742039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:47.742039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:47.742039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:47.742039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:47.742039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:47.742039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:47.742039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:47.745257 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 341/684] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:47.752853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e9600-2c70-4cdc-9115-b0978a78e883 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:47.765456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:47.835992 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c8c8f-0486-414f-9c71-ff2525d6900d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:47.890362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ff942-1abb-47c0-860d-381bbc491ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:47.891351 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ff942-1abb-47c0-860d-381bbc491ccc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:47.891836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ff942-1abb-47c0-860d-381bbc491ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:47.892362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ff942-1abb-47c0-860d-381bbc491ccc None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:47.893314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ff942-1abb-47c0-860d-381bbc491ccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:47.894400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ff942-1abb-47c0-860d-381bbc491ccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:47.896320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ff942-1abb-47c0-860d-381bbc491ccc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:47.909089 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e9600-2c70-4cdc-9115-b0978a78e883 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:47.913680 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 344/685] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:47.919926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ff942-1abb-47c0-860d-381bbc491ccc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:47.956573 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcf88d-4350-4496-b182-96debe202869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:47.957904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcf88d-4350-4496-b182-96debe202869 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:47.958423 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcf88d-4350-4496-b182-96debe202869 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:47.958895 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcf88d-4350-4496-b182-96debe202869 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:47.959799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdcf88d-4350-4496-b182-96debe202869 None admin] RBAC: Adding query filter params (name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:47.960340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdcf88d-4350-4496-b182-96debe202869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:47.961951 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdcf88d-4350-4496-b182-96debe202869 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:47.978229 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 342/686] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:47.980443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcf88d-4350-4496-b182-96debe202869 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:48.042892 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:48.054059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ff942-1abb-47c0-860d-381bbc491ccc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:48.056715 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:48.104212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcf88d-4350-4496-b182-96debe202869 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:48.104989 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781ba17-7f8b-470c-910e-a42bc5acf751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:48.106490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781ba17-7f8b-470c-910e-a42bc5acf751 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:48.107081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781ba17-7f8b-470c-910e-a42bc5acf751 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:48.108297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781ba17-7f8b-470c-910e-a42bc5acf751 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:48.117209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781ba17-7f8b-470c-910e-a42bc5acf751 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:48.119134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781ba17-7f8b-470c-910e-a42bc5acf751 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-f781ba17-7f8b-470c-910e-a42bc5acf751 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:48.136754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:48.137935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:48.139014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:48.139014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:48.139014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:48.139014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:48.139014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:48.139014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:48.139014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:48.139014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:48.139014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:23:48.139014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:48.141892 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 345/687] 10.222.0.62 () {56 vars in 1093 bytes} [Thu Sep 1 16:23:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:23:48.148234 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed55de-75e1-464e-936e-140297d60f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:48.148542 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781ba17-7f8b-470c-910e-a42bc5acf751 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:48.150877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed55de-75e1-464e-936e-140297d60f6d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:48.151340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed55de-75e1-464e-936e-140297d60f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:48.151800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed55de-75e1-464e-936e-140297d60f6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:48.163940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed55de-75e1-464e-936e-140297d60f6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:48.165908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed55de-75e1-464e-936e-140297d60f6d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-abed55de-75e1-464e-936e-140297d60f6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:48.181600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:48.183038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:48.184832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:48.184832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:48.184832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:48.191316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 343/688] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:23:48.192892 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed55de-75e1-464e-936e-140297d60f6d None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:48.273128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781ba17-7f8b-470c-910e-a42bc5acf751 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:48.319955 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5fbfe-060a-42b3-b4d8-6b736eced753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:48.320995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5fbfe-060a-42b3-b4d8-6b736eced753 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:48.321280 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5fbfe-060a-42b3-b4d8-6b736eced753 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:48.321645 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5fbfe-060a-42b3-b4d8-6b736eced753 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:48.322670 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5fbfe-060a-42b3-b4d8-6b736eced753 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:48.323062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5fbfe-060a-42b3-b4d8-6b736eced753 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:48.331924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5fbfe-060a-42b3-b4d8-6b736eced753 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:48.346486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 346/689] 10.222.0.62 () {56 vars in 1133 bytes} [Thu Sep 1 16:23:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:48.352197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5fbfe-060a-42b3-b4d8-6b736eced753 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:48.358420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed55de-75e1-464e-936e-140297d60f6d None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:48.364859 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:48.406993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:48.445874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b50f8-5ca6-419d-9a17-7b42b4cdb43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:48.446509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b50f8-5ca6-419d-9a17-7b42b4cdb43e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:48.446864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b50f8-5ca6-419d-9a17-7b42b4cdb43e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:48.447156 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b50f8-5ca6-419d-9a17-7b42b4cdb43e None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:48.449533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b50f8-5ca6-419d-9a17-7b42b4cdb43e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:48.449851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b50f8-5ca6-419d-9a17-7b42b4cdb43e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:48.451399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b50f8-5ca6-419d-9a17-7b42b4cdb43e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:48.465725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5fbfe-060a-42b3-b4d8-6b736eced753 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:48.492102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 344/690] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:48.492102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b50f8-5ca6-419d-9a17-7b42b4cdb43e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:48.552046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbe016-a8fe-4212-bf18-dad6a3dc7d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:48.553096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbe016-a8fe-4212-bf18-dad6a3dc7d39 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:48.553594 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbe016-a8fe-4212-bf18-dad6a3dc7d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:48.554057 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbe016-a8fe-4212-bf18-dad6a3dc7d39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:48.614058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b50f8-5ca6-419d-9a17-7b42b4cdb43e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:48.650981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbe016-a8fe-4212-bf18-dad6a3dc7d39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:48.652488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da412792-c498-4ec1-ad7e-392eac97124a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:48.653101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbe016-a8fe-4212-bf18-dad6a3dc7d39 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:48.653209 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da412792-c498-4ec1-ad7e-392eac97124a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:48.653456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da412792-c498-4ec1-ad7e-392eac97124a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:48.653645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da412792-c498-4ec1-ad7e-392eac97124a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:48.662321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da412792-c498-4ec1-ad7e-392eac97124a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:48.663873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da412792-c498-4ec1-ad7e-392eac97124a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:48.666230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-da412792-c498-4ec1-ad7e-392eac97124a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:48.678988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:48.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:48.680949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:48.680949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:48.680949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:48.680949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:48.680949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:48.680949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:48.680949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:48.680949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:48.680949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:23:48.680949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:48.680949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 345/691] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:23:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:48.681716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2cc31-881b-4ecf-9961-7bbf8e00a2b5 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:48.682298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2cc31-881b-4ecf-9961-7bbf8e00a2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:48.682298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2cc31-881b-4ecf-9961-7bbf8e00a2b5 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:48.683708 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 346/692] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:48.686735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2cc31-881b-4ecf-9961-7bbf8e00a2b5 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:48.727871 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:48.774765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 347/693] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:48.776441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a85d2-d9f4-4aea-b144-aadbfa6be110 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:48.776960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a85d2-d9f4-4aea-b144-aadbfa6be110 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:48.777370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a85d2-d9f4-4aea-b144-aadbfa6be110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:48.796326 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5b7a85d2-d9f4-4aea-b144-aadbfa6be110 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:48.797842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5b7a85d2-d9f4-4aea-b144-aadbfa6be110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:48.802473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2cc31-881b-4ecf-9961-7bbf8e00a2b5 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:48.847062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa8340-72f3-4f16-874a-188a4caa6cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:48.848434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa8340-72f3-4f16-874a-188a4caa6cce None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:48.848981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa8340-72f3-4f16-874a-188a4caa6cce None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:48.849726 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa8340-72f3-4f16-874a-188a4caa6cce None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:48.851169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa8340-72f3-4f16-874a-188a4caa6cce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:48.851788 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa8340-72f3-4f16-874a-188a4caa6cce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:48.854769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa8340-72f3-4f16-874a-188a4caa6cce None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:48.872876 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 348/694] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:48.874024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a85d2-d9f4-4aea-b144-aadbfa6be110 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:48.884397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 347/695] 10.222.0.62 () {56 vars in 1135 bytes} [Thu Sep 1 16:23:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:48.893609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa8340-72f3-4f16-874a-188a4caa6cce None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:48.919211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:48.982628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a85d2-d9f4-4aea-b144-aadbfa6be110 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:49.028056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b8703-5ff9-4b33-9d63-f15a9264ed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:49.028056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b8703-5ff9-4b33-9d63-f15a9264ed66 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:49.028056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b8703-5ff9-4b33-9d63-f15a9264ed66 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:49.028056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b8703-5ff9-4b33-9d63-f15a9264ed66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:49.033402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b8703-5ff9-4b33-9d63-f15a9264ed66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:49.035469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b8703-5ff9-4b33-9d63-f15a9264ed66 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-f28b8703-5ff9-4b33-9d63-f15a9264ed66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:49.050979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:49.052049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:49.052980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:49.052980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:49.052980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:49.052980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 349/696] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:49.057983 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b8703-5ff9-4b33-9d63-f15a9264ed66 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:49.133709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa8340-72f3-4f16-874a-188a4caa6cce None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:49.136157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:49.189597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4007c0-4d92-47d1-9c0a-71e55268559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:49.190126 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4007c0-4d92-47d1-9c0a-71e55268559b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:49.190446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4007c0-4d92-47d1-9c0a-71e55268559b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:49.190718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4007c0-4d92-47d1-9c0a-71e55268559b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:49.207605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:49.220534 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b8703-5ff9-4b33-9d63-f15a9264ed66 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:49.277935 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4007c0-4d92-47d1-9c0a-71e55268559b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:49.279444 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4007c0-4d92-47d1-9c0a-71e55268559b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:49.282592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3637f-d8bf-4b0b-a519-249c0ede2c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:49.283204 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3637f-d8bf-4b0b-a519-249c0ede2c9b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:49.283477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3637f-d8bf-4b0b-a519-249c0ede2c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:49.283742 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3637f-d8bf-4b0b-a519-249c0ede2c9b None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:49.284416 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3637f-d8bf-4b0b-a519-249c0ede2c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:49.284716 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3637f-d8bf-4b0b-a519-249c0ede2c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:49.286207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3637f-d8bf-4b0b-a519-249c0ede2c9b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:49.299501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 350/697] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:49.304580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3637f-d8bf-4b0b-a519-249c0ede2c9b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:49.393194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:49.403560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 348/698] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:23:49.404917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4007c0-4d92-47d1-9c0a-71e55268559b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:49.411727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3637f-d8bf-4b0b-a519-249c0ede2c9b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:49.448246 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd27ac-21c7-4529-88be-d39dac136116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:49.448720 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd27ac-21c7-4529-88be-d39dac136116 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:49.448993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd27ac-21c7-4529-88be-d39dac136116 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:49.449258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd27ac-21c7-4529-88be-d39dac136116 None admin] PATH_INFO: `/v3/services/network` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:49.449893 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd27ac-21c7-4529-88be-d39dac136116 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:49.451266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd27ac-21c7-4529-88be-d39dac136116 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-4bcd27ac-21c7-4529-88be-d39dac136116 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:49.462675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:49.463946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:49.464790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 351/699] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:49 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:49.465584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd27ac-21c7-4529-88be-d39dac136116 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:49.483254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:49.530309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4007c0-4d92-47d1-9c0a-71e55268559b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:49.568308 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ecb46-22a9-457c-abb8-0d68239ae38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:49.569329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd27ac-21c7-4529-88be-d39dac136116 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:49.569567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ecb46-22a9-457c-abb8-0d68239ae38a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:49.570036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ecb46-22a9-457c-abb8-0d68239ae38a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:49.570510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ecb46-22a9-457c-abb8-0d68239ae38a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:49.578407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ecb46-22a9-457c-abb8-0d68239ae38a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:49.580289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ecb46-22a9-457c-abb8-0d68239ae38a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-3d4ecb46-22a9-457c-abb8-0d68239ae38a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:49.593525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:49.594425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:49.595323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:49.595323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:49.595323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:49.595323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:49.595323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:49.595323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:49.595323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:49.595323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:49.598793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 349/700] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:49.599793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ecb46-22a9-457c-abb8-0d68239ae38a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:49.609338 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5962a-63b9-4158-84fc-2389fcc4f676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:49.609768 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5962a-63b9-4158-84fc-2389fcc4f676 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:49.610042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5962a-63b9-4158-84fc-2389fcc4f676 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:49.610307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5962a-63b9-4158-84fc-2389fcc4f676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:49.622072 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5962a-63b9-4158-84fc-2389fcc4f676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:49.623532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5962a-63b9-4158-84fc-2389fcc4f676 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-1bb5962a-63b9-4158-84fc-2389fcc4f676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:49.638269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:49.639230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:49.640192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:49.640192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:49.640192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:49.640192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 352/701] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:49.641081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5962a-63b9-4158-84fc-2389fcc4f676 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:49.666015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:49.707704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:49.718483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ecb46-22a9-457c-abb8-0d68239ae38a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:49.743277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5962a-63b9-4158-84fc-2389fcc4f676 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:49.772929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef67a9-0f08-49bf-81b9-9d5708f0738e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:49.774526 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef67a9-0f08-49bf-81b9-9d5708f0738e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:49.775656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef67a9-0f08-49bf-81b9-9d5708f0738e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:49.776187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef67a9-0f08-49bf-81b9-9d5708f0738e None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:49.777228 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef67a9-0f08-49bf-81b9-9d5708f0738e None admin] RBAC: Adding query filter params (name=network) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:49.777864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef67a9-0f08-49bf-81b9-9d5708f0738e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:49.779534 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f481057-3024-4a31-b0ff-7d7295e81554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:49.779683 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef67a9-0f08-49bf-81b9-9d5708f0738e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:49.780265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f481057-3024-4a31-b0ff-7d7295e81554 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:49.780609 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f481057-3024-4a31-b0ff-7d7295e81554 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:49.780945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f481057-3024-4a31-b0ff-7d7295e81554 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:49.781725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f481057-3024-4a31-b0ff-7d7295e81554 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:49.782114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f481057-3024-4a31-b0ff-7d7295e81554 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:49.783639 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f481057-3024-4a31-b0ff-7d7295e81554 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:49.792763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 350/702] 10.222.0.62 () {56 vars in 1112 bytes} [Thu Sep 1 16:23:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:49.795377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef67a9-0f08-49bf-81b9-9d5708f0738e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:49.804418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 353/703] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:49.805605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f481057-3024-4a31-b0ff-7d7295e81554 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:49.924017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f481057-3024-4a31-b0ff-7d7295e81554 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:49.930002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef67a9-0f08-49bf-81b9-9d5708f0738e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:49.950270 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:49.956663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:49.967486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045a34d-badf-479d-ba43-0daa456ed5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:49.968054 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045a34d-badf-479d-ba43-0daa456ed5cf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:49.968326 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045a34d-badf-479d-ba43-0daa456ed5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:49.968591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045a34d-badf-479d-ba43-0daa456ed5cf None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:49.969222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045a34d-badf-479d-ba43-0daa456ed5cf None admin] RBAC: Adding query filter params (type=network) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:49.969417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f4f6f-a070-4d7b-a384-fd8ffabc5bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:49.969549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045a34d-badf-479d-ba43-0daa456ed5cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:49.970031 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f4f6f-a070-4d7b-a384-fd8ffabc5bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:49.970307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f4f6f-a070-4d7b-a384-fd8ffabc5bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:49.970577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f4f6f-a070-4d7b-a384-fd8ffabc5bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:49.970853 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045a34d-badf-479d-ba43-0daa456ed5cf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:49.971213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f4f6f-a070-4d7b-a384-fd8ffabc5bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:49.971511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f4f6f-a070-4d7b-a384-fd8ffabc5bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:49.972910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f4f6f-a070-4d7b-a384-fd8ffabc5bf7 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:49.984241 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 354/704] 10.222.0.62 () {56 vars in 1112 bytes} [Thu Sep 1 16:23:49 2022] GET /identity/v3/services?type=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:49.984241 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045a34d-badf-479d-ba43-0daa456ed5cf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:49.992384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 351/705] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:49.994754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8077d9e-5a27-4839-bfb7-38831b481fa8 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:49.995049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8077d9e-5a27-4839-bfb7-38831b481fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:49.995327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8077d9e-5a27-4839-bfb7-38831b481fa8 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:49.997067 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 352/706] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:49.997862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8077d9e-5a27-4839-bfb7-38831b481fa8 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:50.114491 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045a34d-badf-479d-ba43-0daa456ed5cf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:50.151688 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8077d9e-5a27-4839-bfb7-38831b481fa8 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:50.160136 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caea70-5ae4-4dc0-a175-923efda4ea2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:50.160835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caea70-5ae4-4dc0-a175-923efda4ea2d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:50.161114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caea70-5ae4-4dc0-a175-923efda4ea2d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:50.161384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caea70-5ae4-4dc0-a175-923efda4ea2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:50.162119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3caea70-5ae4-4dc0-a175-923efda4ea2d None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=2912c02ff6db4cbc97c80cd923238136) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:50.162429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3caea70-5ae4-4dc0-a175-923efda4ea2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:50.163853 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3caea70-5ae4-4dc0-a175-923efda4ea2d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:50.171083 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 355/707] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:49 2022] GET /identity/v3/role_assignments?user.id=2912c02ff6db4cbc97c80cd923238136&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:50.172338 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c66dd-4a52-49ce-a239-2cefebaec48a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:50.172622 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c66dd-4a52-49ce-a239-2cefebaec48a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:50.172894 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c66dd-4a52-49ce-a239-2cefebaec48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:50.191844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcf2b7-7e3b-42b3-9dee-6bd0dcfe3156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:50.192526 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcf2b7-7e3b-42b3-9dee-6bd0dcfe3156 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:50.192801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcf2b7-7e3b-42b3-9dee-6bd0dcfe3156 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:50.193068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcf2b7-7e3b-42b3-9dee-6bd0dcfe3156 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:50.193415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1a7c66dd-4a52-49ce-a239-2cefebaec48a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:50.194882 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1a7c66dd-4a52-49ce-a239-2cefebaec48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:50.203377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcf2b7-7e3b-42b3-9dee-6bd0dcfe3156 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:50.204898 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcf2b7-7e3b-42b3-9dee-6bd0dcfe3156 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-9edcf2b7-7e3b-42b3-9dee-6bd0dcfe3156 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:50.223696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:50.224686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:50.225645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:50.225645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:50.225645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:50.225645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:50.225645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:50.225645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:50.225645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:50.225645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:50.231061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 353/708] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:50.233100 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcf2b7-7e3b-42b3-9dee-6bd0dcfe3156 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:50.288153 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:50.303252 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:50.334966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 356/709] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:50.336574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a773439-12bf-4d43-b5d1-98d20cef390a None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:50.337029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a773439-12bf-4d43-b5d1-98d20cef390a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:50.337615 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a773439-12bf-4d43-b5d1-98d20cef390a None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:50.339417 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 357/710] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:50.349793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11615b3a-eb5b-4bff-b80d-2d565f01e7c1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:50.350067 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11615b3a-eb5b-4bff-b80d-2d565f01e7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:50.350333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11615b3a-eb5b-4bff-b80d-2d565f01e7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:50.389140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-11615b3a-eb5b-4bff-b80d-2d565f01e7c1 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:50.390877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-11615b3a-eb5b-4bff-b80d-2d565f01e7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:50.397289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcf2b7-7e3b-42b3-9dee-6bd0dcfe3156 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:50.436288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256f578-43aa-44ad-b679-17789f027dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:50.436909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256f578-43aa-44ad-b679-17789f027dfa None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:50.437180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256f578-43aa-44ad-b679-17789f027dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:50.437448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256f578-43aa-44ad-b679-17789f027dfa None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:50.438137 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256f578-43aa-44ad-b679-17789f027dfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:50.438434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256f578-43aa-44ad-b679-17789f027dfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:50.440002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256f578-43aa-44ad-b679-17789f027dfa None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:50.460100 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 354/711] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:50.463681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256f578-43aa-44ad-b679-17789f027dfa None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:50.482823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 358/712] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:50.626683 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:50.630768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256f578-43aa-44ad-b679-17789f027dfa None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:50.679293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df4a10-89f5-48b7-bef2-8fe06633e2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:50.679913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df4a10-89f5-48b7-bef2-8fe06633e2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:50.680194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df4a10-89f5-48b7-bef2-8fe06633e2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:50.680455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df4a10-89f5-48b7-bef2-8fe06633e2a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:50.681344 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df4a10-89f5-48b7-bef2-8fe06633e2a9 None admin] RBAC: Adding query filter params (role.id=fc3db8095f31464ca4c378645d1d078b, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=56f9bc7262b64bbab0f670576c6bdfdd) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:50.681678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df4a10-89f5-48b7-bef2-8fe06633e2a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:50.683108 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df4a10-89f5-48b7-bef2-8fe06633e2a9 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:50.687931 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11615b3a-eb5b-4bff-b80d-2d565f01e7c1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:50.689434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 355/713] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:50 2022] GET /identity/v3/role_assignments?user.id=56f9bc7262b64bbab0f670576c6bdfdd&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=fc3db8095f31464ca4c378645d1d078b => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:50.842106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11615b3a-eb5b-4bff-b80d-2d565f01e7c1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:50.850939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df4a10-89f5-48b7-bef2-8fe06633e2a9 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:50.860429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:50.887308 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e717a04-31b0-4842-8d4d-0d00d7a25f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:50.887997 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e717a04-31b0-4842-8d4d-0d00d7a25f38 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:50.888275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e717a04-31b0-4842-8d4d-0d00d7a25f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:50.888541 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e717a04-31b0-4842-8d4d-0d00d7a25f38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:50.905258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e717a04-31b0-4842-8d4d-0d00d7a25f38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:50.908317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e717a04-31b0-4842-8d4d-0d00d7a25f38 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-7e717a04-31b0-4842-8d4d-0d00d7a25f38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:50.927283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:50.928349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:50.929234 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:50.929234 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:50.929234 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:50.934166 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 359/714] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:50.935499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e717a04-31b0-4842-8d4d-0d00d7a25f38 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:51.042713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df4a10-89f5-48b7-bef2-8fe06633e2a9 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:51.060933 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e717a04-31b0-4842-8d4d-0d00d7a25f38 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:51.090141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dc61e-32ca-47d0-bd64-9386ad36dcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:51.090792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc61e-32ca-47d0-bd64-9386ad36dcc5 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:51.091071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc61e-32ca-47d0-bd64-9386ad36dcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:51.091342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc61e-32ca-47d0-bd64-9386ad36dcc5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:51.092220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7dc61e-32ca-47d0-bd64-9386ad36dcc5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:51.094327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7dc61e-32ca-47d0-bd64-9386ad36dcc5 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:51.102161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03c5aa-e82b-40ec-b0d0-d1ed50b35b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:51.102767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c5aa-e82b-40ec-b0d0-d1ed50b35b26 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:51.103092 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c5aa-e82b-40ec-b0d0-d1ed50b35b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:51.103363 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c5aa-e82b-40ec-b0d0-d1ed50b35b26 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:51.104239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03c5aa-e82b-40ec-b0d0-d1ed50b35b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:51.104602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03c5aa-e82b-40ec-b0d0-d1ed50b35b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:51.107085 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03c5aa-e82b-40ec-b0d0-d1ed50b35b26 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-0a7dc61e-32ca-47d0-bd64-9386ad36dcc5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:51.108786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:51.109778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:51.110646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:51.110646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:51.110646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:51.110646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:51.110646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:51.110646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:51.110646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:51.110646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:51.110646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:51.110646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:51.110646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:23:51.110646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:23:51.110646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:23:51.110646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:51.111419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 356/715] 10.222.0.62 () {56 vars in 1129 bytes} [Thu Sep 1 16:23:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:51.118195 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dc61e-32ca-47d0-bd64-9386ad36dcc5 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:51.131986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 360/716] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:51.136522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03c5aa-e82b-40ec-b0d0-d1ed50b35b26 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:51.155709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:51.176918 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:51.248606 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dc61e-32ca-47d0-bd64-9386ad36dcc5 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:51.273403 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03c5aa-e82b-40ec-b0d0-d1ed50b35b26 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:51.289131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e71c2-24b6-4470-8d3c-42723cddf5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:51.290255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e71c2-24b6-4470-8d3c-42723cddf5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:51.290845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e71c2-24b6-4470-8d3c-42723cddf5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:51.291184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e71c2-24b6-4470-8d3c-42723cddf5c0 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:51.292174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e71c2-24b6-4470-8d3c-42723cddf5c0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:51.292469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e71c2-24b6-4470-8d3c-42723cddf5c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:51.293897 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e71c2-24b6-4470-8d3c-42723cddf5c0 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:51.298984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 357/717] 10.222.0.62 () {56 vars in 1124 bytes} [Thu Sep 1 16:23:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:51.304505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e71c2-24b6-4470-8d3c-42723cddf5c0 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:51.314154 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e95a35-820a-44c4-af5d-4f079bd38d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:51.315184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e95a35-820a-44c4-af5d-4f079bd38d75 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:51.315671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e95a35-820a-44c4-af5d-4f079bd38d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:51.316197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e95a35-820a-44c4-af5d-4f079bd38d75 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:51.317046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e95a35-820a-44c4-af5d-4f079bd38d75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:51.318695 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e95a35-820a-44c4-af5d-4f079bd38d75 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:51.350771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 361/718] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:23:51 2022] POST /identity/v3/users => generated 254 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:23:51.428049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:51.441613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e71c2-24b6-4470-8d3c-42723cddf5c0 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:51.484267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0005430-fe5f-4fc0-9053-358d7666d90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:51.484954 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0005430-fe5f-4fc0-9053-358d7666d90a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:51.485292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0005430-fe5f-4fc0-9053-358d7666d90a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:51.485564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0005430-fe5f-4fc0-9053-358d7666d90a None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:51.486218 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0005430-fe5f-4fc0-9053-358d7666d90a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:51.486577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0005430-fe5f-4fc0-9053-358d7666d90a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:51.491556 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0005430-fe5f-4fc0-9053-358d7666d90a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:51.499171 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 358/719] 10.222.0.62 () {56 vars in 1124 bytes} [Thu Sep 1 16:23:51 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:52.314028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa40ba8-68f5-4219-b67c-0e57e753c07e None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:52.314563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa40ba8-68f5-4219-b67c-0e57e753c07e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:52.314563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa40ba8-68f5-4219-b67c-0e57e753c07e None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:52.320893 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 362/720] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:52.328368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84c84a-2c94-4930-96b9-d688a7961067 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:52.328875 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84c84a-2c94-4930-96b9-d688a7961067 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:52.328972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84c84a-2c94-4930-96b9-d688a7961067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:52.360953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-0f84c84a-2c94-4930-96b9-d688a7961067 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:52.362185 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0f84c84a-2c94-4930-96b9-d688a7961067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:52.450685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 359/721] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:52.474847 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc95790-d861-42e0-b065-3fd2d0077a15 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:52.475149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc95790-d861-42e0-b065-3fd2d0077a15 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:52.475395 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc95790-d861-42e0-b065-3fd2d0077a15 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:52.476880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 363/722] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:52.483150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b83f7-379f-4718-8dca-7eae1d70c496 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:52.483429 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b83f7-379f-4718-8dca-7eae1d70c496 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:52.483702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b83f7-379f-4718-8dca-7eae1d70c496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:52.506414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-214b83f7-379f-4718-8dca-7eae1d70c496 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:52.507649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-214b83f7-379f-4718-8dca-7eae1d70c496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:52.586273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 360/723] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:52.806340 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc95790-d861-42e0-b065-3fd2d0077a15 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:52.923957 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:52.941555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc95790-d861-42e0-b065-3fd2d0077a15 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:52.983524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b83f7-379f-4718-8dca-7eae1d70c496 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:52.985764 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648f104-c53c-4b83-adf3-90e24d2459b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:52.986432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648f104-c53c-4b83-adf3-90e24d2459b4 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:52.986722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648f104-c53c-4b83-adf3-90e24d2459b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:52.987002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648f104-c53c-4b83-adf3-90e24d2459b4 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:52.987694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648f104-c53c-4b83-adf3-90e24d2459b4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:52.989182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648f104-c53c-4b83-adf3-90e24d2459b4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:53.014065 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 364/724] 10.222.0.62 () {60 vars in 1141 bytes} [Thu Sep 1 16:23:52 2022] POST /identity/v3/services => generated 240 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:23:53.015574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:53.159943 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b83f7-379f-4718-8dca-7eae1d70c496 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:53.224067 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1002f-8f83-4221-bb0f-4b9f9957bd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:53.224654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1002f-8f83-4221-bb0f-4b9f9957bd25 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:53.224937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1002f-8f83-4221-bb0f-4b9f9957bd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:53.225208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1002f-8f83-4221-bb0f-4b9f9957bd25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:53.240292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1002f-8f83-4221-bb0f-4b9f9957bd25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-0dd1002f-8f83-4221-bb0f-4b9f9957bd25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:53.245373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:53.246385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:53.247216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:53.247216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:53.247216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:53.247216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:53.247216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:53.247216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:53.247216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:53.247216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:53.247216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:53.247216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:53.247216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:53.247216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:53.247216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 361/725] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:23:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:53.253105 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648f104-c53c-4b83-adf3-90e24d2459b4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:53.356962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8154f97-f130-421b-a226-9655a91ebb86 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:53.357472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8154f97-f130-421b-a226-9655a91ebb86 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:53.357472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8154f97-f130-421b-a226-9655a91ebb86 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:53.359588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 362/726] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:53.365516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dca93-f8df-47a5-9e06-edb9f588f985 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:53.366104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dca93-f8df-47a5-9e06-edb9f588f985 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:53.366579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dca93-f8df-47a5-9e06-edb9f588f985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:53.393909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-191dca93-f8df-47a5-9e06-edb9f588f985 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:53.395518 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-191dca93-f8df-47a5-9e06-edb9f588f985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:53.404732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648f104-c53c-4b83-adf3-90e24d2459b4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:53.446085 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1bfab-b68a-4ee7-8f02-98d894fa3dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:53.446877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1bfab-b68a-4ee7-8f02-98d894fa3dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:53.447177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1bfab-b68a-4ee7-8f02-98d894fa3dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:53.447418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1bfab-b68a-4ee7-8f02-98d894fa3dc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:53.448220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1bfab-b68a-4ee7-8f02-98d894fa3dc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:53.448522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1bfab-b68a-4ee7-8f02-98d894fa3dc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:53.450043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1bfab-b68a-4ee7-8f02-98d894fa3dc9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:53.464365 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 365/727] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:53.467967 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1bfab-b68a-4ee7-8f02-98d894fa3dc9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:53.472861 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:53.501039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:53.515913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 363/728] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:53 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:23:53.607749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1bfab-b68a-4ee7-8f02-98d894fa3dc9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:53.653634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15efc24-d5f4-43f2-b86f-1c40f56126ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:53.654302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15efc24-d5f4-43f2-b86f-1c40f56126ac None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:53.654581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15efc24-d5f4-43f2-b86f-1c40f56126ac None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:53.654855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15efc24-d5f4-43f2-b86f-1c40f56126ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:53.669677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15efc24-d5f4-43f2-b86f-1c40f56126ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:53.671141 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15efc24-d5f4-43f2-b86f-1c40f56126ac None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b15efc24-d5f4-43f2-b86f-1c40f56126ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:53.689932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:53.690944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:53.692462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:53.692462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:53.692462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:53.692462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 366/729] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:53.696962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191dca93-f8df-47a5-9e06-edb9f588f985 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:53.816703 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191dca93-f8df-47a5-9e06-edb9f588f985 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:53.856870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc266f2a-d659-4659-9d05-b380fa9b9bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:53.857495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc266f2a-d659-4659-9d05-b380fa9b9bcd None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:53.857775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc266f2a-d659-4659-9d05-b380fa9b9bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:53.858042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc266f2a-d659-4659-9d05-b380fa9b9bcd None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:53.858738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc266f2a-d659-4659-9d05-b380fa9b9bcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:53.859045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc266f2a-d659-4659-9d05-b380fa9b9bcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:53.860462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc266f2a-d659-4659-9d05-b380fa9b9bcd None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:53.872774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 364/730] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:53.881249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15efc24-d5f4-43f2-b86f-1c40f56126ac None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:53.886241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1814694-74b1-4594-b582-7f2ab721395d None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:53.886535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1814694-74b1-4594-b582-7f2ab721395d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:53.886803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1814694-74b1-4594-b582-7f2ab721395d None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:53.888408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 365/731] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:23:53.889108 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1814694-74b1-4594-b582-7f2ab721395d None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:53.946716 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:53.960437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:53.993460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15efc24-d5f4-43f2-b86f-1c40f56126ac None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:54.006469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1814694-74b1-4594-b582-7f2ab721395d None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:54.039693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5dd332-54e3-4dd2-b21f-6654197b3c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:54.040540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5dd332-54e3-4dd2-b21f-6654197b3c0f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:54.040898 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5dd332-54e3-4dd2-b21f-6654197b3c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:54.041395 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5dd332-54e3-4dd2-b21f-6654197b3c0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:54.042938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eac78c-6eb5-4eaa-8df8-84bd306b7096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:54.043584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eac78c-6eb5-4eaa-8df8-84bd306b7096 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:54.043862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eac78c-6eb5-4eaa-8df8-84bd306b7096 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:54.044138 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eac78c-6eb5-4eaa-8df8-84bd306b7096 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:54.049991 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5dd332-54e3-4dd2-b21f-6654197b3c0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:54.051660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5dd332-54e3-4dd2-b21f-6654197b3c0f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:54.055952 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eac78c-6eb5-4eaa-8df8-84bd306b7096 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-a4eac78c-6eb5-4eaa-8df8-84bd306b7096 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:54.060947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:54.063094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:54.065166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:54.065166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:54.065166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:54.065166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:54.065166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:54.065166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:54.065166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:54.065166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:54.065166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:54.065166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:54.065166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:54.065166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-dc5dd332-54e3-4dd2-b21f-6654197b3c0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:54.066689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:54.076729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:54.079049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:54.079049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:54.079049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:54.079049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:54.079049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:54.079049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:54.079049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:54.079049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:54.079049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:23:54.079049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:54.079049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 367/733] 10.222.0.62 () {56 vars in 1093 bytes} [Thu Sep 1 16:23:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:54.079049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc750559-d84f-47d6-94ca-785a9293d247 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:54.079049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc750559-d84f-47d6-94ca-785a9293d247 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:54.080743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc750559-d84f-47d6-94ca-785a9293d247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:54.080897 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 366/733] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:23:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:54.080897 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eac78c-6eb5-4eaa-8df8-84bd306b7096 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:54.087130 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-fc750559-d84f-47d6-94ca-785a9293d247 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:54.088821 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fc750559-d84f-47d6-94ca-785a9293d247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:54.157913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 368/734] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:54 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:54.158786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc750559-d84f-47d6-94ca-785a9293d247 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:54.197903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eac78c-6eb5-4eaa-8df8-84bd306b7096 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:54.211967 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:54.220432 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:54.236625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ba575-75f1-464e-9195-2fe250876abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:54.237240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ba575-75f1-464e-9195-2fe250876abf None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:54.237518 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ba575-75f1-464e-9195-2fe250876abf None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:54.237786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ba575-75f1-464e-9195-2fe250876abf None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:54.238551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ba575-75f1-464e-9195-2fe250876abf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:54.238856 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ba575-75f1-464e-9195-2fe250876abf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:54.240357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ba575-75f1-464e-9195-2fe250876abf None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:54.249874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 367/735] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:54.251230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc166ff-04cd-45e3-890c-cf0c1b63cbe9 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:54.251516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc166ff-04cd-45e3-890c-cf0c1b63cbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:54.251841 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc166ff-04cd-45e3-890c-cf0c1b63cbe9 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:54.253603 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 368/736] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:54.255341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc166ff-04cd-45e3-890c-cf0c1b63cbe9 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:54.282601 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc750559-d84f-47d6-94ca-785a9293d247 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:54.324321 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2795e-05ff-47e2-a2ef-eea5c9aa8be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:54.325335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2795e-05ff-47e2-a2ef-eea5c9aa8be8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:54.325641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2795e-05ff-47e2-a2ef-eea5c9aa8be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:54.326009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2795e-05ff-47e2-a2ef-eea5c9aa8be8 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:54.326892 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2795e-05ff-47e2-a2ef-eea5c9aa8be8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:54.327315 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2795e-05ff-47e2-a2ef-eea5c9aa8be8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:54.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2795e-05ff-47e2-a2ef-eea5c9aa8be8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:54.387847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc166ff-04cd-45e3-890c-cf0c1b63cbe9 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:54.432159 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a3d4c-a73f-4b73-953c-231120c26f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:54.432818 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a3d4c-a73f-4b73-953c-231120c26f46 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:54.433097 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a3d4c-a73f-4b73-953c-231120c26f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:54.433363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a3d4c-a73f-4b73-953c-231120c26f46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:54.445959 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a3d4c-a73f-4b73-953c-231120c26f46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:54.447911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a3d4c-a73f-4b73-953c-231120c26f46 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-b69a3d4c-a73f-4b73-953c-231120c26f46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:54.464511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:54.465560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:54.466924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:54.466924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:54.466924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:54.466924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 369/737] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:54.467325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18864898-9013-4c67-836f-5a03bb48b5c2 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:54.467473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18864898-9013-4c67-836f-5a03bb48b5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:54.468692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18864898-9013-4c67-836f-5a03bb48b5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:54.474238 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 369/738] 10.222.0.62 () {56 vars in 1133 bytes} [Thu Sep 1 16:23:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:54.475307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2795e-05ff-47e2-a2ef-eea5c9aa8be8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:54.489545 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-18864898-9013-4c67-836f-5a03bb48b5c2 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:54.491102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-18864898-9013-4c67-836f-5a03bb48b5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:54.518300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:54.571915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 370/739] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:54 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:23:54.571915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18864898-9013-4c67-836f-5a03bb48b5c2 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:54.590525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2795e-05ff-47e2-a2ef-eea5c9aa8be8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:54.592952 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:54.627122 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08407b-39d6-49ee-b83a-2c787fdef294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:54.627994 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08407b-39d6-49ee-b83a-2c787fdef294 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:54.628369 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08407b-39d6-49ee-b83a-2c787fdef294 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:54.628738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08407b-39d6-49ee-b83a-2c787fdef294 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:54.629480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b08407b-39d6-49ee-b83a-2c787fdef294 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:54.630948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b08407b-39d6-49ee-b83a-2c787fdef294 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:54.647920 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 370/740] 10.222.0.62 () {60 vars in 1141 bytes} [Thu Sep 1 16:23:54 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:23:54.648902 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08407b-39d6-49ee-b83a-2c787fdef294 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:54.718973 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18864898-9013-4c67-836f-5a03bb48b5c2 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:54.804691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498fb56-a889-4581-9a20-037701330687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:54.805338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498fb56-a889-4581-9a20-037701330687 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:54.805615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498fb56-a889-4581-9a20-037701330687 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:54.805885 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498fb56-a889-4581-9a20-037701330687 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:54.806557 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498fb56-a889-4581-9a20-037701330687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:54.806856 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498fb56-a889-4581-9a20-037701330687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:54.808286 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498fb56-a889-4581-9a20-037701330687 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:54.827018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 371/741] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:54.832240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498fb56-a889-4581-9a20-037701330687 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:54.842462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08407b-39d6-49ee-b83a-2c787fdef294 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:54.864298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:54.887466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e692db-1ae7-4177-9af0-be81e52fd26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:54.889057 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e692db-1ae7-4177-9af0-be81e52fd26f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:54.889552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e692db-1ae7-4177-9af0-be81e52fd26f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:54.890008 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e692db-1ae7-4177-9af0-be81e52fd26f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:54.902925 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e692db-1ae7-4177-9af0-be81e52fd26f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:54.904828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e692db-1ae7-4177-9af0-be81e52fd26f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-96e692db-1ae7-4177-9af0-be81e52fd26f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:54.922232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:54.923226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:54.924163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:54.924163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:54.924163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:54.924163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 371/742] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:54.924163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e692db-1ae7-4177-9af0-be81e52fd26f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:54.945377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:54.959584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498fb56-a889-4581-9a20-037701330687 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:54.996183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148daccc-067a-4449-88cc-b1dfa01e920e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:54.997205 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148daccc-067a-4449-88cc-b1dfa01e920e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:54.997706 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148daccc-067a-4449-88cc-b1dfa01e920e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:54.998152 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148daccc-067a-4449-88cc-b1dfa01e920e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:55.006697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148daccc-067a-4449-88cc-b1dfa01e920e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:55.008362 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148daccc-067a-4449-88cc-b1dfa01e920e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-148daccc-067a-4449-88cc-b1dfa01e920e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:55.021220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:55.022417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:55.023377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:55.023377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:55.023377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:55.023377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:55.023377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:55.023377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:55.023377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:55.023377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:55.023377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:23:55.023377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:55.023960 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 372/743] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:23:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:23:55.024409 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148daccc-067a-4449-88cc-b1dfa01e920e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:55.035539 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e692db-1ae7-4177-9af0-be81e52fd26f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:55.080855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334fec9-0597-414b-94f3-4729db95ea06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:55.081528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334fec9-0597-414b-94f3-4729db95ea06 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:55.081793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334fec9-0597-414b-94f3-4729db95ea06 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:55.082069 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334fec9-0597-414b-94f3-4729db95ea06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:55.094228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334fec9-0597-414b-94f3-4729db95ea06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-6334fec9-0597-414b-94f3-4729db95ea06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:55.099046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:55.100078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:55.101004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:55.101004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:55.101004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:55.101004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:55.101004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:55.101004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:55.101004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:55.101004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:55.101004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:55.101004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:55.101004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:23:55.101004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:55.104032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 372/744] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:23:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:55.104941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334fec9-0597-414b-94f3-4729db95ea06 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:55.126791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:55.158762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148daccc-067a-4449-88cc-b1dfa01e920e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:55.191908 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:55.207106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e77ffe-b049-46b8-bbab-e38d32ca82c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:55.208802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e77ffe-b049-46b8-bbab-e38d32ca82c9 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:55.209523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e77ffe-b049-46b8-bbab-e38d32ca82c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:55.210020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e77ffe-b049-46b8-bbab-e38d32ca82c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:55.211463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e77ffe-b049-46b8-bbab-e38d32ca82c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:55.212131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e77ffe-b049-46b8-bbab-e38d32ca82c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:55.214722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e77ffe-b049-46b8-bbab-e38d32ca82c9 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:55.217562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334fec9-0597-414b-94f3-4729db95ea06 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:55.232147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 373/745] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:55.234036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e77ffe-b049-46b8-bbab-e38d32ca82c9 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:55.256184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428f9e1-3de1-4965-a52b-f225ce3349d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:55.256880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428f9e1-3de1-4965-a52b-f225ce3349d0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:55.257170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428f9e1-3de1-4965-a52b-f225ce3349d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:55.257435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428f9e1-3de1-4965-a52b-f225ce3349d0 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:55.258275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428f9e1-3de1-4965-a52b-f225ce3349d0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:55.258580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428f9e1-3de1-4965-a52b-f225ce3349d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:55.260468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428f9e1-3de1-4965-a52b-f225ce3349d0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:55.280124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 373/746] 10.222.0.62 () {56 vars in 1135 bytes} [Thu Sep 1 16:23:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:55.280800 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428f9e1-3de1-4965-a52b-f225ce3349d0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:55.368321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e77ffe-b049-46b8-bbab-e38d32ca82c9 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:55.384032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428f9e1-3de1-4965-a52b-f225ce3349d0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:55.406664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2db1d3-8b7b-452f-bc3d-6bd754995f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:55.407336 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2db1d3-8b7b-452f-bc3d-6bd754995f9b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:55.407692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2db1d3-8b7b-452f-bc3d-6bd754995f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:55.408007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2db1d3-8b7b-452f-bc3d-6bd754995f9b None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:55.408766 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2db1d3-8b7b-452f-bc3d-6bd754995f9b None admin] RBAC: Adding query filter params (name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:55.409066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2db1d3-8b7b-452f-bc3d-6bd754995f9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:55.410535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2db1d3-8b7b-452f-bc3d-6bd754995f9b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:55.419048 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0003a26-1788-4960-a794-446eaeac86c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:55.419676 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0003a26-1788-4960-a794-446eaeac86c3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:55.420075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 374/747] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:23:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:55.420965 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2db1d3-8b7b-452f-bc3d-6bd754995f9b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:55.423663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:55.424355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0003a26-1788-4960-a794-446eaeac86c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:55.424657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0003a26-1788-4960-a794-446eaeac86c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:55.427126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:55.435007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0003a26-1788-4960-a794-446eaeac86c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:55.436859 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0003a26-1788-4960-a794-446eaeac86c3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a0003a26-1788-4960-a794-446eaeac86c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:55.449812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:55.450743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:55.451628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:55.451628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:55.451628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:55.451628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:55.451628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:55.451628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:55.451628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:55.451628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:55.452522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 374/748] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:55.453450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0003a26-1788-4960-a794-446eaeac86c3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:55.528449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2db1d3-8b7b-452f-bc3d-6bd754995f9b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:55.562174 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0003a26-1788-4960-a794-446eaeac86c3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:55.563067 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22f4e6-15af-473a-89b0-1892eacd668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:55.563667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22f4e6-15af-473a-89b0-1892eacd668e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:55.563969 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22f4e6-15af-473a-89b0-1892eacd668e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:55.564243 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22f4e6-15af-473a-89b0-1892eacd668e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:55.601078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fcf67-bcc0-4b87-ae68-34b63eef24fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:55.601815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fcf67-bcc0-4b87-ae68-34b63eef24fc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:55.602091 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fcf67-bcc0-4b87-ae68-34b63eef24fc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:55.602357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fcf67-bcc0-4b87-ae68-34b63eef24fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:55.614667 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155fcf67-bcc0-4b87-ae68-34b63eef24fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:55.616296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155fcf67-bcc0-4b87-ae68-34b63eef24fc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-155fcf67-bcc0-4b87-ae68-34b63eef24fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:55.632178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:55.633288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:55.634533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:55.634533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:55.634533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:55.635814 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22f4e6-15af-473a-89b0-1892eacd668e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:55.637307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22f4e6-15af-473a-89b0-1892eacd668e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:55.638891 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 375/749] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:55.639255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:55.641026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a87dd5-1105-4276-827c-727082b171de None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:55.641329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a87dd5-1105-4276-827c-727082b171de None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:55.641563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a87dd5-1105-4276-827c-727082b171de None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:55.642851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 376/750] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:55.644708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a87dd5-1105-4276-827c-727082b171de None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:55.698779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:55.745072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 375/751] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:55 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:23:55.746234 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22f4e6-15af-473a-89b0-1892eacd668e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:55.767658 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a87dd5-1105-4276-827c-727082b171de None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:55.806534 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009daeb6-630c-47b2-8d56-8cd137efac7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:55.808265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009daeb6-630c-47b2-8d56-8cd137efac7b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:55.808770 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009daeb6-630c-47b2-8d56-8cd137efac7b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:55.809393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009daeb6-630c-47b2-8d56-8cd137efac7b None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:55.810746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009daeb6-630c-47b2-8d56-8cd137efac7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:55.811332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009daeb6-630c-47b2-8d56-8cd137efac7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:55.813594 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009daeb6-630c-47b2-8d56-8cd137efac7b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:55.827156 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 377/752] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:55.828233 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945cdae-bec4-4204-933e-bc8e33e646a3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:55.828505 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945cdae-bec4-4204-933e-bc8e33e646a3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:55.828764 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945cdae-bec4-4204-933e-bc8e33e646a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:55.847430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6945cdae-bec4-4204-933e-bc8e33e646a3 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:55.848662 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6945cdae-bec4-4204-933e-bc8e33e646a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:55.859221 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22f4e6-15af-473a-89b0-1892eacd668e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:55.861244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:55.896970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816da18-e9db-4505-9d03-991c3da25928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:55.897517 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816da18-e9db-4505-9d03-991c3da25928 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:55.897804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816da18-e9db-4505-9d03-991c3da25928 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:55.898069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816da18-e9db-4505-9d03-991c3da25928 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:55.898703 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816da18-e9db-4505-9d03-991c3da25928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:55.898998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816da18-e9db-4505-9d03-991c3da25928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:55.901197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816da18-e9db-4505-9d03-991c3da25928 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:55.913047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 376/753] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:55.918646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816da18-e9db-4505-9d03-991c3da25928 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:55.920604 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 378/754] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:23:55.921555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945cdae-bec4-4204-933e-bc8e33e646a3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:55.976221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:56.040386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816da18-e9db-4505-9d03-991c3da25928 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:56.059975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945cdae-bec4-4204-933e-bc8e33e646a3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:56.084141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edb1ac-615b-4f39-813a-a996b9373d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:56.084870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edb1ac-615b-4f39-813a-a996b9373d3d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:56.085202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edb1ac-615b-4f39-813a-a996b9373d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:56.086056 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edb1ac-615b-4f39-813a-a996b9373d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:56.100418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edb1ac-615b-4f39-813a-a996b9373d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:56.101949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edb1ac-615b-4f39-813a-a996b9373d3d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:56.102451 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee354d48-b37d-4ca5-8891-e5e7ccfc927a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:56.103331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354d48-b37d-4ca5-8891-e5e7ccfc927a None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:56.103849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354d48-b37d-4ca5-8891-e5e7ccfc927a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:56.104190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354d48-b37d-4ca5-8891-e5e7ccfc927a None admin] PATH_INFO: `/v3/projects/425048ffd13448cd8fe815333a9b798c/users/2912c02ff6db4cbc97c80cd923238136/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:56.106524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee354d48-b37d-4ca5-8891-e5e7ccfc927a None admin] RBAC: Authorizing `identity:create_grant(project_id=425048ffd13448cd8fe815333a9b798c, user_id=2912c02ff6db4cbc97c80cd923238136, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:56.107977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee354d48-b37d-4ca5-8891-e5e7ccfc927a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-41edb1ac-615b-4f39-813a-a996b9373d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:56.119852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:56.120826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:56.121720 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:56.121720 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:56.121720 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:56.121720 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 377/755] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:56.123756 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 379/756] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:23:55 2022] PUT /identity/v3/projects/425048ffd13448cd8fe815333a9b798c/users/2912c02ff6db4cbc97c80cd923238136/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:23:56.126935 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edb1ac-615b-4f39-813a-a996b9373d3d None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:56.128982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee354d48-b37d-4ca5-8891-e5e7ccfc927a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:56.163862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:56.288962 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:56.296177 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edb1ac-615b-4f39-813a-a996b9373d3d None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:56.299386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee354d48-b37d-4ca5-8891-e5e7ccfc927a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:56.336091 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5c0cc-0bf5-410c-bfed-b0609e311e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:56.337006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6318e0-9cd3-471c-b3c6-71705103b4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:56.337164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5c0cc-0bf5-410c-bfed-b0609e311e48 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:56.337640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5c0cc-0bf5-410c-bfed-b0609e311e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:56.337973 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6318e0-9cd3-471c-b3c6-71705103b4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:56.338216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5c0cc-0bf5-410c-bfed-b0609e311e48 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:56.338323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6318e0-9cd3-471c-b3c6-71705103b4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:56.338646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6318e0-9cd3-471c-b3c6-71705103b4b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:56.339311 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6318e0-9cd3-471c-b3c6-71705103b4b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:56.339608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6318e0-9cd3-471c-b3c6-71705103b4b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:56.341097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6318e0-9cd3-471c-b3c6-71705103b4b8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:56.349953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5c0cc-0bf5-410c-bfed-b0609e311e48 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:56.351936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5c0cc-0bf5-410c-bfed-b0609e311e48 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:56.355795 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 380/757] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:56.356112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6318e0-9cd3-471c-b3c6-71705103b4b8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-53f5c0cc-0bf5-410c-bfed-b0609e311e48 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:23:56.366026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:56.367047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:56.368104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:56.368104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:56.368104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:23:56.368104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:23:56.368104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:23:56.368104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:23:56.368104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:23:56.368104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:23:56.368104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:23:56.368104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:56.368104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 378/758] 10.222.0.62 () {56 vars in 1117 bytes} [Thu Sep 1 16:23:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:56.372822 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5c0cc-0bf5-410c-bfed-b0609e311e48 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:56.473325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:56.481581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6318e0-9cd3-471c-b3c6-71705103b4b8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:56.502219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5c0cc-0bf5-410c-bfed-b0609e311e48 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:56.522442 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37cbb8-bdf0-4e0c-b737-14da6b0697a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:56.523188 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37cbb8-bdf0-4e0c-b737-14da6b0697a2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:56.523583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37cbb8-bdf0-4e0c-b737-14da6b0697a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:56.523927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37cbb8-bdf0-4e0c-b737-14da6b0697a2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:56.524625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37cbb8-bdf0-4e0c-b737-14da6b0697a2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:56.526150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37cbb8-bdf0-4e0c-b737-14da6b0697a2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ec37cbb8-bdf0-4e0c-b737-14da6b0697a2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:56.535666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:56.536748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:56.537617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 381/759] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:56 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:56.538448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37cbb8-bdf0-4e0c-b737-14da6b0697a2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:56.548579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b2ad7-b949-47b9-8d6c-40901f01f4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:56.550065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b2ad7-b949-47b9-8d6c-40901f01f4fe None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:56.550552 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b2ad7-b949-47b9-8d6c-40901f01f4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:56.551476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b2ad7-b949-47b9-8d6c-40901f01f4fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:56.561998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b2ad7-b949-47b9-8d6c-40901f01f4fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:56.563874 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:56.565985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b2ad7-b949-47b9-8d6c-40901f01f4fe None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-261b2ad7-b949-47b9-8d6c-40901f01f4fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:56.583919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:56.584870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:56.585762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:56.585762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:56.585762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:56.585762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:56.585762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:56.585762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:56.585762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:56.585762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:56.593556 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 379/760] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:56.593556 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b2ad7-b949-47b9-8d6c-40901f01f4fe None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:56.664531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37cbb8-bdf0-4e0c-b737-14da6b0697a2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:56.715464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c5a4a-074e-4654-8a4a-dfd2dd2b486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:56.716435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c5a4a-074e-4654-8a4a-dfd2dd2b486e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:56.717077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c5a4a-074e-4654-8a4a-dfd2dd2b486e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:56.717915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c5a4a-074e-4654-8a4a-dfd2dd2b486e None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:56.719154 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c5a4a-074e-4654-8a4a-dfd2dd2b486e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:56.719666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c5a4a-074e-4654-8a4a-dfd2dd2b486e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:56.720811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b2ad7-b949-47b9-8d6c-40901f01f4fe None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:56.721478 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c5a4a-074e-4654-8a4a-dfd2dd2b486e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:56.746840 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 382/761] 10.222.0.62 () {56 vars in 1149 bytes} [Thu Sep 1 16:23:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:56.748165 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c5a4a-074e-4654-8a4a-dfd2dd2b486e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:56.757768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66f429-09a3-4895-a70a-55ed60078722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:56.758530 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66f429-09a3-4895-a70a-55ed60078722 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:56.758903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66f429-09a3-4895-a70a-55ed60078722 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:56.759475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66f429-09a3-4895-a70a-55ed60078722 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:56.760563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66f429-09a3-4895-a70a-55ed60078722 None admin] RBAC: Adding query filter params (name=network) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:56.760690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66f429-09a3-4895-a70a-55ed60078722 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:56.762392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66f429-09a3-4895-a70a-55ed60078722 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:56.764118 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:56.769896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 380/762] 10.222.0.62 () {56 vars in 1112 bytes} [Thu Sep 1 16:23:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:56.771116 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66f429-09a3-4895-a70a-55ed60078722 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:56.843834 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:56.866862 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c5a4a-074e-4654-8a4a-dfd2dd2b486e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:56.882078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66f429-09a3-4895-a70a-55ed60078722 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:56.903271 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c95d98-e1f4-40a3-a50f-cb51ae0670f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:56.903938 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c95d98-e1f4-40a3-a50f-cb51ae0670f8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:56.904225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c95d98-e1f4-40a3-a50f-cb51ae0670f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:56.904491 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c95d98-e1f4-40a3-a50f-cb51ae0670f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:56.905226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c95d98-e1f4-40a3-a50f-cb51ae0670f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:56.905625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c95d98-e1f4-40a3-a50f-cb51ae0670f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:56.907088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c95d98-e1f4-40a3-a50f-cb51ae0670f8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:56.919010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 383/763] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:56.919461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc990cd5-e3f5-4eb9-95a6-b9a8dc06f79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:56.919823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c95d98-e1f4-40a3-a50f-cb51ae0670f8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:56.920579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc990cd5-e3f5-4eb9-95a6-b9a8dc06f79f None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:56.920991 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc990cd5-e3f5-4eb9-95a6-b9a8dc06f79f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:56.921381 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc990cd5-e3f5-4eb9-95a6-b9a8dc06f79f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:56.993096 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:56.997479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc990cd5-e3f5-4eb9-95a6-b9a8dc06f79f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:56.999501 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc990cd5-e3f5-4eb9-95a6-b9a8dc06f79f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:57.025238 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c95d98-e1f4-40a3-a50f-cb51ae0670f8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:57.059718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944866d-72fb-4bf3-b15d-b34c5438d467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:57.060373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944866d-72fb-4bf3-b15d-b34c5438d467 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:57.060656 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944866d-72fb-4bf3-b15d-b34c5438d467 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:57.060926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944866d-72fb-4bf3-b15d-b34c5438d467 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:57.061560 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944866d-72fb-4bf3-b15d-b34c5438d467 None admin] RBAC: Adding query filter params (type=network) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:57.061858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944866d-72fb-4bf3-b15d-b34c5438d467 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:57.063235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944866d-72fb-4bf3-b15d-b34c5438d467 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:57.068015 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 384/764] 10.222.0.62 () {56 vars in 1112 bytes} [Thu Sep 1 16:23:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:57.069157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405dc7e-045f-4b3c-a692-f1d3fa63a7b0 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:57.069437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405dc7e-045f-4b3c-a692-f1d3fa63a7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:57.069703 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405dc7e-045f-4b3c-a692-f1d3fa63a7b0 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:57.071065 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 385/765] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:57.071566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405dc7e-045f-4b3c-a692-f1d3fa63a7b0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:57.078837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:57.117408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 381/766] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:23:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:23:57.117408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc990cd5-e3f5-4eb9-95a6-b9a8dc06f79f None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:57.195911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405dc7e-045f-4b3c-a692-f1d3fa63a7b0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:57.215753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:57.220188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc990cd5-e3f5-4eb9-95a6-b9a8dc06f79f None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:57.234301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb86edc-40b9-4fa2-a39f-a144dc911c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:57.234919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb86edc-40b9-4fa2-a39f-a144dc911c87 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:57.235364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb86edc-40b9-4fa2-a39f-a144dc911c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:57.235637 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb86edc-40b9-4fa2-a39f-a144dc911c87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:57.236802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb86edc-40b9-4fa2-a39f-a144dc911c87 None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=56f9bc7262b64bbab0f670576c6bdfdd) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:57.237297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb86edc-40b9-4fa2-a39f-a144dc911c87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:57.238690 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb86edc-40b9-4fa2-a39f-a144dc911c87 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:57.246448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 386/767] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:57 2022] GET /identity/v3/role_assignments?user.id=56f9bc7262b64bbab0f670576c6bdfdd&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:57.247549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6e56b-edd6-4a4d-ab84-878dddb08a3e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:57.247809 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6e56b-edd6-4a4d-ab84-878dddb08a3e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:57.248086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6e56b-edd6-4a4d-ab84-878dddb08a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:57.269084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-aea6e56b-edd6-4a4d-ab84-878dddb08a3e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:57.270344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-aea6e56b-edd6-4a4d-ab84-878dddb08a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:57.274923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf70c4-2abe-4200-97aa-46c1b31e769c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:57.276091 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf70c4-2abe-4200-97aa-46c1b31e769c None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:57.276569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf70c4-2abe-4200-97aa-46c1b31e769c None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:57.277033 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf70c4-2abe-4200-97aa-46c1b31e769c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:57.277959 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf70c4-2abe-4200-97aa-46c1b31e769c None admin] RBAC: Adding query filter params (interface=public, service_id=72640fe04d5f4f6eb725913365debb49, region_id=RegionOne) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:57.278506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf70c4-2abe-4200-97aa-46c1b31e769c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:57.281083 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf70c4-2abe-4200-97aa-46c1b31e769c None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:57.293822 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 382/768] 10.222.0.62 () {56 vars in 1251 bytes} [Thu Sep 1 16:23:57 2022] GET /identity/v3/endpoints?service_id=72640fe04d5f4f6eb725913365debb49&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:57.294504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf70c4-2abe-4200-97aa-46c1b31e769c None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:57.350973 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:57.367958 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 387/769] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:23:57.367958 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6e56b-edd6-4a4d-ab84-878dddb08a3e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:57.464293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf70c4-2abe-4200-97aa-46c1b31e769c None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:57.494446 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6e56b-edd6-4a4d-ab84-878dddb08a3e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:57.499472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d7c05-4ee9-4ec4-b81d-494d7ec4f2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:57.500071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d7c05-4ee9-4ec4-b81d-494d7ec4f2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:57.500342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d7c05-4ee9-4ec4-b81d-494d7ec4f2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:57.500604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d7c05-4ee9-4ec4-b81d-494d7ec4f2e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:57.514231 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d7c05-4ee9-4ec4-b81d-494d7ec4f2e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:57.514231 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d7c05-4ee9-4ec4-b81d-494d7ec4f2e6 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-ae0d7c05-4ee9-4ec4-b81d-494d7ec4f2e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:57.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:57.534862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:57.535783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:57.535783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:57.535783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:57.535783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 383/770] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:57.548015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d7c05-4ee9-4ec4-b81d-494d7ec4f2e6 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:57.569230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa883d-c5b7-4a64-bf57-d537756a3f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:57.569909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa883d-c5b7-4a64-bf57-d537756a3f81 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:57.570125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa883d-c5b7-4a64-bf57-d537756a3f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:57.570395 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa883d-c5b7-4a64-bf57-d537756a3f81 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:57.571162 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa883d-c5b7-4a64-bf57-d537756a3f81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:57.572631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa883d-c5b7-4a64-bf57-d537756a3f81 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:57.594947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 388/771] 10.222.0.62 () {56 vars in 1087 bytes} [Thu Sep 1 16:23:57 2022] GET /identity/v3/services => generated 772 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:57.597879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:57.671171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa883d-c5b7-4a64-bf57-d537756a3f81 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:57.729606 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d7c05-4ee9-4ec4-b81d-494d7ec4f2e6 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:57.768339 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:57.812381 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8317e5-bae1-4458-8b87-4f7fc6af88a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:57.813474 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8317e5-bae1-4458-8b87-4f7fc6af88a7 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:57.813969 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8317e5-bae1-4458-8b87-4f7fc6af88a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:57.814453 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8317e5-bae1-4458-8b87-4f7fc6af88a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:57.815314 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8317e5-bae1-4458-8b87-4f7fc6af88a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:57.815817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8317e5-bae1-4458-8b87-4f7fc6af88a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:57.817480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8317e5-bae1-4458-8b87-4f7fc6af88a7 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:57.842455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 384/772] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:57.847155 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8317e5-bae1-4458-8b87-4f7fc6af88a7 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:57.847406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa883d-c5b7-4a64-bf57-d537756a3f81 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:57.889255 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6a590-cf38-4350-b936-4d52e5b448e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:57.890281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6a590-cf38-4350-b936-4d52e5b448e4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:57.890704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6a590-cf38-4350-b936-4d52e5b448e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:57.891119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6a590-cf38-4350-b936-4d52e5b448e4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:57.891871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6a590-cf38-4350-b936-4d52e5b448e4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:57.893617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6a590-cf38-4350-b936-4d52e5b448e4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-1fc6a590-cf38-4350-b936-4d52e5b448e4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:57.907799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:57.909103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:57.910178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:57.910178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:57.910178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:57.910178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:57.910178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:57.910178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:57.910178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:57.910178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:57.910178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:57.910178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:57.910178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:23:57.910178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:23:57.910178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:23:57.910178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:23:57.912801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 389/773] 10.222.0.62 () {56 vars in 1129 bytes} [Thu Sep 1 16:23:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:57.919150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6a590-cf38-4350-b936-4d52e5b448e4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:57.939034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:57.987493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8317e5-bae1-4458-8b87-4f7fc6af88a7 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:58.059386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c796c1c-62da-48ec-a163-a2ec174e7979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:58.059960 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c796c1c-62da-48ec-a163-a2ec174e7979 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:58.060241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c796c1c-62da-48ec-a163-a2ec174e7979 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:58.060545 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c796c1c-62da-48ec-a163-a2ec174e7979 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:58.070628 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c796c1c-62da-48ec-a163-a2ec174e7979 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:58.080337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c796c1c-62da-48ec-a163-a2ec174e7979 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:58.089153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6a590-cf38-4350-b936-4d52e5b448e4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-3c796c1c-62da-48ec-a163-a2ec174e7979 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:23:58.105213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:58.106208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:58.107094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:23:58.107094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:23:58.107094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:23:58.107094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:23:58.107094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:23:58.107094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:23:58.107094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:23:58.107094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:58.107094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 385/774] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:23:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:58.114441 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c796c1c-62da-48ec-a163-a2ec174e7979 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:58.126819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:58.142155 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd27dac-1a99-4a0a-b418-cc51c577773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:58.142812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd27dac-1a99-4a0a-b418-cc51c577773a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:58.143084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd27dac-1a99-4a0a-b418-cc51c577773a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:58.143344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd27dac-1a99-4a0a-b418-cc51c577773a None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:58.143992 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd27dac-1a99-4a0a-b418-cc51c577773a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:58.144288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd27dac-1a99-4a0a-b418-cc51c577773a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:58.145657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd27dac-1a99-4a0a-b418-cc51c577773a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:58.161932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 390/775] 10.222.0.62 () {56 vars in 1124 bytes} [Thu Sep 1 16:23:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:58.168083 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd27dac-1a99-4a0a-b418-cc51c577773a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:58.242747 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c796c1c-62da-48ec-a163-a2ec174e7979 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:58.285443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac816812-8547-430b-b826-b47ff82af312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:58.286763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac816812-8547-430b-b826-b47ff82af312 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:58.287493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac816812-8547-430b-b826-b47ff82af312 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:58.288412 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac816812-8547-430b-b826-b47ff82af312 None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:58.289608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac816812-8547-430b-b826-b47ff82af312 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:58.290370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac816812-8547-430b-b826-b47ff82af312 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:58.292780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac816812-8547-430b-b826-b47ff82af312 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:58.294981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:58.308485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 386/776] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:23:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:58.310239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac22ae3-64d1-47e8-b940-40c8ebbb40dc None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:58.310665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac22ae3-64d1-47e8-b940-40c8ebbb40dc None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:58.311115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac22ae3-64d1-47e8-b940-40c8ebbb40dc None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:58.316750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 387/777] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:58.317732 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac22ae3-64d1-47e8-b940-40c8ebbb40dc None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:58.343634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd27dac-1a99-4a0a-b418-cc51c577773a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:58.391317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e618737-2489-48f2-8da8-456cb96bb951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:58.392214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e618737-2489-48f2-8da8-456cb96bb951 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:58.392573 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e618737-2489-48f2-8da8-456cb96bb951 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:58.392936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e618737-2489-48f2-8da8-456cb96bb951 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:58.393651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e618737-2489-48f2-8da8-456cb96bb951 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:58.394522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e618737-2489-48f2-8da8-456cb96bb951 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:58.396315 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e618737-2489-48f2-8da8-456cb96bb951 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:58.407585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 391/778] 10.222.0.62 () {56 vars in 1124 bytes} [Thu Sep 1 16:23:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:58.407585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5ebec-c06a-4369-a9ce-108ab30976c5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:58.407585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5ebec-c06a-4369-a9ce-108ab30976c5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:58.407585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5ebec-c06a-4369-a9ce-108ab30976c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:58.427878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-eef5ebec-c06a-4369-a9ce-108ab30976c5 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:58.429405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eef5ebec-c06a-4369-a9ce-108ab30976c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:58.441387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac22ae3-64d1-47e8-b940-40c8ebbb40dc None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:58.493478 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba704fbc-dc1f-48da-ae6b-2ef3fdbc4620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:58.493785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:58.495202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba704fbc-dc1f-48da-ae6b-2ef3fdbc4620 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:58.495550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba704fbc-dc1f-48da-ae6b-2ef3fdbc4620 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:58.495797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba704fbc-dc1f-48da-ae6b-2ef3fdbc4620 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:58.496583 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba704fbc-dc1f-48da-ae6b-2ef3fdbc4620 None admin] RBAC: Adding query filter params (role.id=fc3db8095f31464ca4c378645d1d078b, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=24705faaf5054802a556749c7ba82ee8) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:58.496888 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba704fbc-dc1f-48da-ae6b-2ef3fdbc4620 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:58.498346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba704fbc-dc1f-48da-ae6b-2ef3fdbc4620 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:58.504686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 388/779] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:23:58 2022] GET /identity/v3/role_assignments?user.id=24705faaf5054802a556749c7ba82ee8&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=fc3db8095f31464ca4c378645d1d078b => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:58.505849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba704fbc-dc1f-48da-ae6b-2ef3fdbc4620 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:58.529300 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 392/780] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:23:58.617265 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:58.676188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba704fbc-dc1f-48da-ae6b-2ef3fdbc4620 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:58.713421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0b562-b8eb-417b-a7d5-9f26c487f78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:58.714034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0b562-b8eb-417b-a7d5-9f26c487f78a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:58.714308 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0b562-b8eb-417b-a7d5-9f26c487f78a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:58.714572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0b562-b8eb-417b-a7d5-9f26c487f78a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:58.715266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0b562-b8eb-417b-a7d5-9f26c487f78a None admin] RBAC: Adding query filter params (interface=public, service_id=146a121d2d324778b2d02d074449c2bf, region_id=RegionOne) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:58.715573 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0b562-b8eb-417b-a7d5-9f26c487f78a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:58.716969 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0b562-b8eb-417b-a7d5-9f26c487f78a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:58.726818 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 389/781] 10.222.0.62 () {56 vars in 1251 bytes} [Thu Sep 1 16:23:58 2022] GET /identity/v3/endpoints?service_id=146a121d2d324778b2d02d074449c2bf&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:58.728747 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5ebec-c06a-4369-a9ce-108ab30976c5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:58.876187 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5ebec-c06a-4369-a9ce-108ab30976c5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:58.927031 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2278ea1-d922-45e0-8f77-0c4ed4f2a184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:58.927647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2278ea1-d922-45e0-8f77-0c4ed4f2a184 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:58.927942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2278ea1-d922-45e0-8f77-0c4ed4f2a184 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:58.928216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2278ea1-d922-45e0-8f77-0c4ed4f2a184 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:58.928827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2278ea1-d922-45e0-8f77-0c4ed4f2a184 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:58.930190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2278ea1-d922-45e0-8f77-0c4ed4f2a184 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:58.938916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0b562-b8eb-417b-a7d5-9f26c487f78a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:58.946305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 393/782] 10.222.0.62 () {56 vars in 1087 bytes} [Thu Sep 1 16:23:58 2022] GET /identity/v3/services => generated 772 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:23:59.053184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0b562-b8eb-417b-a7d5-9f26c487f78a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:59.085727 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:59.099215 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca1b5e-d471-479c-b1ad-daea08f00664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:59.100053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca1b5e-d471-479c-b1ad-daea08f00664 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:59.100335 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca1b5e-d471-479c-b1ad-daea08f00664 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:59.100603 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca1b5e-d471-479c-b1ad-daea08f00664 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:59.111639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca1b5e-d471-479c-b1ad-daea08f00664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-56ca1b5e-d471-479c-b1ad-daea08f00664 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:23:59.116687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:59.117668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:59.119252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:59.119252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:59.119252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:59.119252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:59.119252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:23:59.119252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:23:59.119252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:23:59.119252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:23:59.119252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:23:59.119252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:23:59.119252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:23:59.119252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:59.119252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 390/783] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:23:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:23:59.129116 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2278ea1-d922-45e0-8f77-0c4ed4f2a184 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:59.211274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:59.285873 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2278ea1-d922-45e0-8f77-0c4ed4f2a184 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:59.325439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a413549-ac19-40b9-b043-f6b2d5ae6f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:59.326491 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a413549-ac19-40b9-b043-f6b2d5ae6f16 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:59.326963 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a413549-ac19-40b9-b043-f6b2d5ae6f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:59.327433 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a413549-ac19-40b9-b043-f6b2d5ae6f16 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:59.328354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a413549-ac19-40b9-b043-f6b2d5ae6f16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:59.328848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a413549-ac19-40b9-b043-f6b2d5ae6f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:59.330423 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a413549-ac19-40b9-b043-f6b2d5ae6f16 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:59.340256 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 394/784] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:23:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:59.346299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca1b5e-d471-479c-b1ad-daea08f00664 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:59.466425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca1b5e-d471-479c-b1ad-daea08f00664 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:59.509853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d1d2a-9b04-4023-a9ca-8850548310e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:59.510679 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d1d2a-9b04-4023-a9ca-8850548310e3 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:59.510957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d1d2a-9b04-4023-a9ca-8850548310e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:59.511229 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d1d2a-9b04-4023-a9ca-8850548310e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:59.525291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d1d2a-9b04-4023-a9ca-8850548310e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:59.526773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d1d2a-9b04-4023-a9ca-8850548310e3 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-050d1d2a-9b04-4023-a9ca-8850548310e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:23:59.545868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:23:59.546916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:23:59.548516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:23:59.548516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:23:59.548516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:23:59.548516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 391/785] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:23:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:23:59.553357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a413549-ac19-40b9-b043-f6b2d5ae6f16 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:59.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6df44d-4c7b-4567-bc02-5186cf219df4 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:59.570510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6df44d-4c7b-4567-bc02-5186cf219df4 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:59.570794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6df44d-4c7b-4567-bc02-5186cf219df4 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:59.573625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 392/786] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:59.577823 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2bce9-804b-404a-8f2b-d73ade55f8d8 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:59.578121 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2bce9-804b-404a-8f2b-d73ade55f8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:59.578370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2bce9-804b-404a-8f2b-d73ade55f8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:59.598540 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-0df2bce9-804b-404a-8f2b-d73ade55f8d8 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:59.599750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0df2bce9-804b-404a-8f2b-d73ade55f8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:23:59.643932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:59.720851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 393/787] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:23:59.730421 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a413549-ac19-40b9-b043-f6b2d5ae6f16 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:23:59.755143 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:23:59.829265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0cdd0-e9aa-4a9c-9197-e63e60440754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:23:59.829915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0cdd0-e9aa-4a9c-9197-e63e60440754 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:59.830193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0cdd0-e9aa-4a9c-9197-e63e60440754 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:59.830458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0cdd0-e9aa-4a9c-9197-e63e60440754 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:59.831125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0cdd0-e9aa-4a9c-9197-e63e60440754 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:23:59.831427 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0cdd0-e9aa-4a9c-9197-e63e60440754 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:23:59.833169 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0cdd0-e9aa-4a9c-9197-e63e60440754 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:23:59.853076 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 395/788] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:23:59.858728 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2bce9-804b-404a-8f2b-d73ade55f8d8 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:23:59.899716 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef292b9-24a1-4cbc-9644-b3816e3e31a0 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:59.900809 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef292b9-24a1-4cbc-9644-b3816e3e31a0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:59.901101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef292b9-24a1-4cbc-9644-b3816e3e31a0 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:59.902610 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 396/789] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:23:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:23:59.909501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac27bc-490c-4023-b05b-af8bd768a8b9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:23:59.910981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac27bc-490c-4023-b05b-af8bd768a8b9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:23:59.912243 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac27bc-490c-4023-b05b-af8bd768a8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:23:59.943628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-60ac27bc-490c-4023-b05b-af8bd768a8b9 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:23:59.945332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-60ac27bc-490c-4023-b05b-af8bd768a8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:00.039071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2bce9-804b-404a-8f2b-d73ade55f8d8 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:00.059716 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 397/790] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:23:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:00.060812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac27bc-490c-4023-b05b-af8bd768a8b9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:00.081783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4026f15-4248-4249-b0a8-fded081c7015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:00.083016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4026f15-4248-4249-b0a8-fded081c7015 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:00.083436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4026f15-4248-4249-b0a8-fded081c7015 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:00.083846 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4026f15-4248-4249-b0a8-fded081c7015 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:00.092808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4026f15-4248-4249-b0a8-fded081c7015 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:00.094707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4026f15-4248-4249-b0a8-fded081c7015 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-b4026f15-4248-4249-b0a8-fded081c7015 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:00.108169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:00.109309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:00.110427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:00.110427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:00.110427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:00.110427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:00.110427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:00.110427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:00.110427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:00.110427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:00.110427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:24:00.110427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:00.113252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 394/791] 10.222.0.62 () {56 vars in 1093 bytes} [Thu Sep 1 16:23:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:00.118502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4026f15-4248-4249-b0a8-fded081c7015 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:00.148536 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:00.273141 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:00.279946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac27bc-490c-4023-b05b-af8bd768a8b9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:00.285448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4026f15-4248-4249-b0a8-fded081c7015 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:00.343612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f3f6b-8610-407b-a55c-2f3691aab203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:00.344496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21ccc7-59ad-4805-a27b-2b669213de2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:00.345132 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21ccc7-59ad-4805-a27b-2b669213de2b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:00.345407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21ccc7-59ad-4805-a27b-2b669213de2b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:00.345680 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21ccc7-59ad-4805-a27b-2b669213de2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:00.346539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f3f6b-8610-407b-a55c-2f3691aab203 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:00.346807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f3f6b-8610-407b-a55c-2f3691aab203 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:00.347068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f3f6b-8610-407b-a55c-2f3691aab203 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:00.347799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f3f6b-8610-407b-a55c-2f3691aab203 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:00.348124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f3f6b-8610-407b-a55c-2f3691aab203 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:00.349567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f3f6b-8610-407b-a55c-2f3691aab203 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:00.361978 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21ccc7-59ad-4805-a27b-2b669213de2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ad21ccc7-59ad-4805-a27b-2b669213de2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:00.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:00.367870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:00.368858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:00.368858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:00.368858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:00.368858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:00.368858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:00.368858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:00.368858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:00.368858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:00.368858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:00.368858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:00.368858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:00.368858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:00.372379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 398/792] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:24:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:00.374814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21ccc7-59ad-4805-a27b-2b669213de2b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:00.397144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 395/793] 10.222.0.62 () {56 vars in 1133 bytes} [Thu Sep 1 16:24:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:00.403343 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f3f6b-8610-407b-a55c-2f3691aab203 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:00.512390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21ccc7-59ad-4805-a27b-2b669213de2b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:00.522271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:00.545404 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f3f6b-8610-407b-a55c-2f3691aab203 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:00.552489 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8119c3-8994-43fa-9329-880ef4d0a2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:00.553348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8119c3-8994-43fa-9329-880ef4d0a2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:00.553817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8119c3-8994-43fa-9329-880ef4d0a2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:00.554284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8119c3-8994-43fa-9329-880ef4d0a2f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:00.555212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8119c3-8994-43fa-9329-880ef4d0a2f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:00.555708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8119c3-8994-43fa-9329-880ef4d0a2f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:00.557757 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8119c3-8994-43fa-9329-880ef4d0a2f3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:00.571816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 399/794] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:24:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:00.573042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8119c3-8994-43fa-9329-880ef4d0a2f3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:00.586693 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd73706-de18-4b15-9fd5-06ae12ba6ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:00.587558 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd73706-de18-4b15-9fd5-06ae12ba6ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:00.588256 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd73706-de18-4b15-9fd5-06ae12ba6ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:00.588528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd73706-de18-4b15-9fd5-06ae12ba6ec9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:00.597229 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:00.664651 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd73706-de18-4b15-9fd5-06ae12ba6ec9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:00.666820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd73706-de18-4b15-9fd5-06ae12ba6ec9 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:00.698107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8119c3-8994-43fa-9329-880ef4d0a2f3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:00.739724 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f2cc8-b1ea-41d4-824e-c0ce7fdcdecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:00.740438 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f2cc8-b1ea-41d4-824e-c0ce7fdcdecb None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:00.740702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f2cc8-b1ea-41d4-824e-c0ce7fdcdecb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:00.740974 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f2cc8-b1ea-41d4-824e-c0ce7fdcdecb None admin] PATH_INFO: `/v3/services/network` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:00.741565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f2cc8-b1ea-41d4-824e-c0ce7fdcdecb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:00.743139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f2cc8-b1ea-41d4-824e-c0ce7fdcdecb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-5b2f2cc8-b1ea-41d4-824e-c0ce7fdcdecb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:00.758364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:00.759566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:00.760953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 400/795] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:00 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:00.762076 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f2cc8-b1ea-41d4-824e-c0ce7fdcdecb None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:00.774703 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:00.798591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 396/796] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:24:00.799770 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd73706-de18-4b15-9fd5-06ae12ba6ec9 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:00.866602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:00.883965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f2cc8-b1ea-41d4-824e-c0ce7fdcdecb None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:00.912913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd73706-de18-4b15-9fd5-06ae12ba6ec9 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:00.921902 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837d309-f0b2-4354-912b-00db94085e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:00.922541 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837d309-f0b2-4354-912b-00db94085e61 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:00.922818 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837d309-f0b2-4354-912b-00db94085e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:00.923291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837d309-f0b2-4354-912b-00db94085e61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:00.934339 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837d309-f0b2-4354-912b-00db94085e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:00.935921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837d309-f0b2-4354-912b-00db94085e61 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:00.950154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb38506-e11b-4682-9c45-07f1cc6665ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:00.950766 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb38506-e11b-4682-9c45-07f1cc6665ac None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:00.951042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb38506-e11b-4682-9c45-07f1cc6665ac None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:00.951309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb38506-e11b-4682-9c45-07f1cc6665ac None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b837d309-f0b2-4354-912b-00db94085e61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:00.951600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:00.952531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:00.953400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:00.953400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:00.953400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:00.953579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb38506-e11b-4682-9c45-07f1cc6665ac None admin] RBAC: Adding query filter params (name=network) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:00.953579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb38506-e11b-4682-9c45-07f1cc6665ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:00.953702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 401/797] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:00.954274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf0cb8-92c1-4593-bee6-d1f081f662dd None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:00.954557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf0cb8-92c1-4593-bee6-d1f081f662dd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:00.954826 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf0cb8-92c1-4593-bee6-d1f081f662dd None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:00.960722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 402/798] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:00.960722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf0cb8-92c1-4593-bee6-d1f081f662dd None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:00.960945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb38506-e11b-4682-9c45-07f1cc6665ac None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:00.965188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 397/799] 10.222.0.62 () {56 vars in 1112 bytes} [Thu Sep 1 16:24:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:00.967917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb38506-e11b-4682-9c45-07f1cc6665ac None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:01.014889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:01.070932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf0cb8-92c1-4593-bee6-d1f081f662dd None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:01.074313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb38506-e11b-4682-9c45-07f1cc6665ac None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:01.109626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1e997-ab47-4355-903c-71be5503bb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:01.110227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1e997-ab47-4355-903c-71be5503bb6e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:01.110504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1e997-ab47-4355-903c-71be5503bb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:01.110775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1e997-ab47-4355-903c-71be5503bb6e None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:01.111409 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1e997-ab47-4355-903c-71be5503bb6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:01.111709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1e997-ab47-4355-903c-71be5503bb6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:01.113121 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1e997-ab47-4355-903c-71be5503bb6e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:01.114255 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17bcb7-b4d0-401a-b2da-8ad46b6da1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:01.115174 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17bcb7-b4d0-401a-b2da-8ad46b6da1fc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:01.116029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17bcb7-b4d0-401a-b2da-8ad46b6da1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:01.116441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17bcb7-b4d0-401a-b2da-8ad46b6da1fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:01.120869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:01.129787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 398/800] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:01.129787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d321864-6115-4cec-ae09-4566c2cf6a6e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:01.129787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d321864-6115-4cec-ae09-4566c2cf6a6e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:01.129787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d321864-6115-4cec-ae09-4566c2cf6a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:01.136787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17bcb7-b4d0-401a-b2da-8ad46b6da1fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:01.138387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17bcb7-b4d0-401a-b2da-8ad46b6da1fc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:01.145406 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-2d321864-6115-4cec-ae09-4566c2cf6a6e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:01.146602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2d321864-6115-4cec-ae09-4566c2cf6a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-6b17bcb7-b4d0-401a-b2da-8ad46b6da1fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:01.157024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:01.157972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:01.158863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:01.158863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:01.158863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:01.158863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 403/801] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:01.159765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17bcb7-b4d0-401a-b2da-8ad46b6da1fc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:01.225190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 399/802] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:01.226532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d321864-6115-4cec-ae09-4566c2cf6a6e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:01.276789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:01.288552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17bcb7-b4d0-401a-b2da-8ad46b6da1fc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:01.325473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75505e48-d02e-4393-8313-3f11b4f819db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:01.326110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75505e48-d02e-4393-8313-3f11b4f819db None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:01.326388 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75505e48-d02e-4393-8313-3f11b4f819db None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:01.326659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75505e48-d02e-4393-8313-3f11b4f819db None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:01.327288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75505e48-d02e-4393-8313-3f11b4f819db None admin] RBAC: Adding query filter params (type=network) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:01.327590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75505e48-d02e-4393-8313-3f11b4f819db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:01.329024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75505e48-d02e-4393-8313-3f11b4f819db None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:01.334985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 404/803] 10.222.0.62 () {56 vars in 1112 bytes} [Thu Sep 1 16:24:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:01.335861 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75505e48-d02e-4393-8313-3f11b4f819db None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:01.361926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d321864-6115-4cec-ae09-4566c2cf6a6e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:01.396610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375d6b1-8736-47f5-9729-404d7eedf859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:01.396790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:01.397164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375d6b1-8736-47f5-9729-404d7eedf859 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:01.397440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375d6b1-8736-47f5-9729-404d7eedf859 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:01.397792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375d6b1-8736-47f5-9729-404d7eedf859 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:01.406036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375d6b1-8736-47f5-9729-404d7eedf859 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:01.407495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375d6b1-8736-47f5-9729-404d7eedf859 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-5375d6b1-8736-47f5-9729-404d7eedf859 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:01.419014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:01.420062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:01.426029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:01.426029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:01.426029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:01.426029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:01.426029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:01.426029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:01.426029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:01.426029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:01.426029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:24:01.426029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:01.426029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 400/804] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:24:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:01.426029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c6184-122c-4a39-8df8-89b391693c44 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:01.426029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c6184-122c-4a39-8df8-89b391693c44 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:01.426879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c6184-122c-4a39-8df8-89b391693c44 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:01.426879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 401/805] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:01.426879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c6184-122c-4a39-8df8-89b391693c44 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:01.456113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75505e48-d02e-4393-8313-3f11b4f819db None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:01.494643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d8a9b-a136-474a-8460-c498a358f4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:01.495269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d8a9b-a136-474a-8460-c498a358f4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:01.495591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d8a9b-a136-474a-8460-c498a358f4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:01.495857 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d8a9b-a136-474a-8460-c498a358f4a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:01.496525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920d8a9b-a136-474a-8460-c498a358f4a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:01.496824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920d8a9b-a136-474a-8460-c498a358f4a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:01.498158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920d8a9b-a136-474a-8460-c498a358f4a8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:01.514532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 405/806] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:01.514532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d8a9b-a136-474a-8460-c498a358f4a8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:01.534609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:01.541781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c6184-122c-4a39-8df8-89b391693c44 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:01.574598 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a8127-2b0e-4064-bf9f-71134f798b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:01.575212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a8127-2b0e-4064-bf9f-71134f798b80 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:01.575491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a8127-2b0e-4064-bf9f-71134f798b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:01.575763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a8127-2b0e-4064-bf9f-71134f798b80 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:01.576497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a8127-2b0e-4064-bf9f-71134f798b80 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:01.577917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a8127-2b0e-4064-bf9f-71134f798b80 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:01.599760 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 402/807] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:24:01 2022] POST /identity/v3/endpoints => generated 337 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:24:01.601085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7b94b-31bd-4bcc-a768-de0544f3f577 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:01.601369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7b94b-31bd-4bcc-a768-de0544f3f577 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:01.601640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7b94b-31bd-4bcc-a768-de0544f3f577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:01.621383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d8a9b-a136-474a-8460-c498a358f4a8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:01.622612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e8a7b94b-31bd-4bcc-a768-de0544f3f577 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:01.624049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e8a7b94b-31bd-4bcc-a768-de0544f3f577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:01.636972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:01.681987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88fa06-2405-468e-9d40-39c026112e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:01.682678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88fa06-2405-468e-9d40-39c026112e49 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:01.682955 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88fa06-2405-468e-9d40-39c026112e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:01.683219 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88fa06-2405-468e-9d40-39c026112e49 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:01.683985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88fa06-2405-468e-9d40-39c026112e49 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:01.684296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88fa06-2405-468e-9d40-39c026112e49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:01.685767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88fa06-2405-468e-9d40-39c026112e49 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:01.704602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 406/808] 10.222.0.62 () {56 vars in 1135 bytes} [Thu Sep 1 16:24:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:01.705564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88fa06-2405-468e-9d40-39c026112e49 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:01.711390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 403/809] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:01.713159 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7b94b-31bd-4bcc-a768-de0544f3f577 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:01.826815 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:01.850036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7b94b-31bd-4bcc-a768-de0544f3f577 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:01.853443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88fa06-2405-468e-9d40-39c026112e49 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:01.888723 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afdcd7-ee60-4d68-8d83-8c844ac043f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:01.888948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e139fc4-db3d-448c-9438-765d3c4eec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:01.889584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e139fc4-db3d-448c-9438-765d3c4eec05 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:01.889861 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e139fc4-db3d-448c-9438-765d3c4eec05 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:01.890128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e139fc4-db3d-448c-9438-765d3c4eec05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:01.890373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afdcd7-ee60-4d68-8d83-8c844ac043f2 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:01.890811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afdcd7-ee60-4d68-8d83-8c844ac043f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:01.891252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afdcd7-ee60-4d68-8d83-8c844ac043f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:01.898608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afdcd7-ee60-4d68-8d83-8c844ac043f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:01.900414 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e139fc4-db3d-448c-9438-765d3c4eec05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:01.900652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afdcd7-ee60-4d68-8d83-8c844ac043f2 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2e139fc4-db3d-448c-9438-765d3c4eec05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:01.905373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:01.906305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:01.907135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:01.907135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:01.907135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:01.907135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:01.907135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:01.907135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:01.907135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:01.907135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:01.907135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:01.907135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:01.907135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:24:01.907135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:01.907135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 407/810] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:24:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:01.907849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e139fc4-db3d-448c-9438-765d3c4eec05 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:01.942466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:02.024995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e139fc4-db3d-448c-9438-765d3c4eec05 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-03afdcd7-ee60-4d68-8d83-8c844ac043f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:02.035925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:02.037024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:02.038220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:02.038220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:02.038220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:02.038220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:02.038220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:02.038220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:02.038220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:02.038220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:02.038220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 404/811] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:02.038220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afdcd7-ee60-4d68-8d83-8c844ac043f2 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:02.063620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bea113-05c2-449c-9287-ce9849db56d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:02.064280 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bea113-05c2-449c-9287-ce9849db56d4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:02.064550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bea113-05c2-449c-9287-ce9849db56d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:02.064818 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bea113-05c2-449c-9287-ce9849db56d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:02.075164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bea113-05c2-449c-9287-ce9849db56d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:02.076652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bea113-05c2-449c-9287-ce9849db56d4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e2bea113-05c2-449c-9287-ce9849db56d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:02.092316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:02.093398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:02.094293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:02.094293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:02.094293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:02.094293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 408/812] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:02.094293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bea113-05c2-449c-9287-ce9849db56d4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:02.145913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afdcd7-ee60-4d68-8d83-8c844ac043f2 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:02.177847 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:02.182180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cc37f-aec1-4ef8-888a-0ea2112356cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:02.182782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cc37f-aec1-4ef8-888a-0ea2112356cd None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:02.183076 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cc37f-aec1-4ef8-888a-0ea2112356cd None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:02.183344 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cc37f-aec1-4ef8-888a-0ea2112356cd None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:02.184042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cc37f-aec1-4ef8-888a-0ea2112356cd None admin] RBAC: Adding query filter params (name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:02.184342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cc37f-aec1-4ef8-888a-0ea2112356cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:02.185733 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cc37f-aec1-4ef8-888a-0ea2112356cd None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:02.194450 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 405/813] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:24:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:02.195015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cc37f-aec1-4ef8-888a-0ea2112356cd None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:02.198676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:02.204194 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bea113-05c2-449c-9287-ce9849db56d4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:02.237430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f616ef-e28c-4b4e-96e7-756aceccfe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:02.238052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f616ef-e28c-4b4e-96e7-756aceccfe0f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:02.238329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f616ef-e28c-4b4e-96e7-756aceccfe0f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:02.238592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f616ef-e28c-4b4e-96e7-756aceccfe0f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:02.239163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f616ef-e28c-4b4e-96e7-756aceccfe0f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:02.240542 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f616ef-e28c-4b4e-96e7-756aceccfe0f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-02f616ef-e28c-4b4e-96e7-756aceccfe0f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:02.249348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:02.250370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:02.251209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 409/814] 10.222.0.62 () {56 vars in 1129 bytes} [Thu Sep 1 16:24:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:02.252014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f616ef-e28c-4b4e-96e7-756aceccfe0f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:02.300922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cc37f-aec1-4ef8-888a-0ea2112356cd None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:02.337841 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48170519-333f-465d-9c77-ec7228dc4301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:02.338436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48170519-333f-465d-9c77-ec7228dc4301 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:02.338709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48170519-333f-465d-9c77-ec7228dc4301 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:02.338973 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48170519-333f-465d-9c77-ec7228dc4301 None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:02.339649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48170519-333f-465d-9c77-ec7228dc4301 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:02.339973 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48170519-333f-465d-9c77-ec7228dc4301 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:02.341395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48170519-333f-465d-9c77-ec7228dc4301 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:02.353625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f616ef-e28c-4b4e-96e7-756aceccfe0f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:02.355338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 406/815] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:02.356171 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48170519-333f-465d-9c77-ec7228dc4301 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:02.388686 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ed84a-510a-4317-94b7-9a1a352d7549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:02.389287 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ed84a-510a-4317-94b7-9a1a352d7549 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:02.389563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ed84a-510a-4317-94b7-9a1a352d7549 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:02.389829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ed84a-510a-4317-94b7-9a1a352d7549 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:02.390459 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ed84a-510a-4317-94b7-9a1a352d7549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:02.390751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ed84a-510a-4317-94b7-9a1a352d7549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:02.392140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ed84a-510a-4317-94b7-9a1a352d7549 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:02.403837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 410/816] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:02.404669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ed84a-510a-4317-94b7-9a1a352d7549 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:02.411761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:02.421433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:02.464403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48170519-333f-465d-9c77-ec7228dc4301 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:02.500627 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f547022-214f-4c95-b556-a086c670c2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:02.501255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f547022-214f-4c95-b556-a086c670c2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:02.501524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f547022-214f-4c95-b556-a086c670c2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:02.501786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f547022-214f-4c95-b556-a086c670c2d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:02.511792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f547022-214f-4c95-b556-a086c670c2d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:02.513266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f547022-214f-4c95-b556-a086c670c2d7 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:02.517713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ed84a-510a-4317-94b7-9a1a352d7549 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-7f547022-214f-4c95-b556-a086c670c2d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:02.528331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:02.529437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:02.530313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:02.530313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:02.530313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:02.530313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 407/817] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:02.530313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f547022-214f-4c95-b556-a086c670c2d7 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:02.554444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697567c-5f30-45dd-9bd7-8bd8f52a14bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:02.555124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697567c-5f30-45dd-9bd7-8bd8f52a14bb None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:02.555407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697567c-5f30-45dd-9bd7-8bd8f52a14bb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:02.555677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697567c-5f30-45dd-9bd7-8bd8f52a14bb None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:02.556623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697567c-5f30-45dd-9bd7-8bd8f52a14bb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:02.556931 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697567c-5f30-45dd-9bd7-8bd8f52a14bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:02.558302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697567c-5f30-45dd-9bd7-8bd8f52a14bb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:02.562711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 411/818] 10.222.0.62 () {56 vars in 1124 bytes} [Thu Sep 1 16:24:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:02.563467 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697567c-5f30-45dd-9bd7-8bd8f52a14bb None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:02.631829 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f547022-214f-4c95-b556-a086c670c2d7 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:02.632381 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:02.650392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:02.669391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef92df0-91be-4d2a-8850-a844c5e6738d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:02.669991 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef92df0-91be-4d2a-8850-a844c5e6738d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:02.670262 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef92df0-91be-4d2a-8850-a844c5e6738d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:02.670524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef92df0-91be-4d2a-8850-a844c5e6738d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:02.670684 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697567c-5f30-45dd-9bd7-8bd8f52a14bb None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:02.671227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef92df0-91be-4d2a-8850-a844c5e6738d None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=2912c02ff6db4cbc97c80cd923238136) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:02.671526 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef92df0-91be-4d2a-8850-a844c5e6738d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:02.673018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef92df0-91be-4d2a-8850-a844c5e6738d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:02.678557 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 408/819] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:02 2022] GET /identity/v3/role_assignments?user.id=2912c02ff6db4cbc97c80cd923238136&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:02.679327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef92df0-91be-4d2a-8850-a844c5e6738d None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:02.715835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf7482-e940-4f9a-8db7-0864d764c188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:02.716429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf7482-e940-4f9a-8db7-0864d764c188 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:02.716726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf7482-e940-4f9a-8db7-0864d764c188 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:02.716990 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf7482-e940-4f9a-8db7-0864d764c188 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:02.724507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf7482-e940-4f9a-8db7-0864d764c188 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:02.725972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf7482-e940-4f9a-8db7-0864d764c188 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-6dbf7482-e940-4f9a-8db7-0864d764c188 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:02.738946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:02.740079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:02.740982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:02.740982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:02.740982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:02.740982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:02.740982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:02.740982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:02.740982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:02.740982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:02.740982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 412/820] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:02.741472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf7482-e940-4f9a-8db7-0864d764c188 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:02.802721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef92df0-91be-4d2a-8850-a844c5e6738d None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:02.848886 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201af23e-a039-4232-bea9-440931344b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:02.849492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201af23e-a039-4232-bea9-440931344b69 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:02.849761 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201af23e-a039-4232-bea9-440931344b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:02.850022 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201af23e-a039-4232-bea9-440931344b69 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:02.851528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201af23e-a039-4232-bea9-440931344b69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:02.851826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201af23e-a039-4232-bea9-440931344b69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:02.853416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201af23e-a039-4232-bea9-440931344b69 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:02.867807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 409/821] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:02.868900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201af23e-a039-4232-bea9-440931344b69 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:02.878614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf7482-e940-4f9a-8db7-0864d764c188 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:02.907151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:02.914626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:02.922092 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c0ee0-2c49-4768-88e8-5fc93a26c43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:02.922822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c0ee0-2c49-4768-88e8-5fc93a26c43b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:02.923100 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c0ee0-2c49-4768-88e8-5fc93a26c43b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:02.923373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c0ee0-2c49-4768-88e8-5fc93a26c43b None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:02.924114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c0ee0-2c49-4768-88e8-5fc93a26c43b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:02.924421 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c0ee0-2c49-4768-88e8-5fc93a26c43b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:02.925890 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c0ee0-2c49-4768-88e8-5fc93a26c43b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:02.931111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 413/822] 10.222.0.62 () {56 vars in 1124 bytes} [Thu Sep 1 16:24:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:02.936384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c0ee0-2c49-4768-88e8-5fc93a26c43b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:02.977384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201af23e-a039-4232-bea9-440931344b69 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:03.014738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd79872-8906-4a99-96eb-6a8887bafeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:03.015434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd79872-8906-4a99-96eb-6a8887bafeff None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:03.015769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd79872-8906-4a99-96eb-6a8887bafeff None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:03.016122 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd79872-8906-4a99-96eb-6a8887bafeff None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:03.016869 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd79872-8906-4a99-96eb-6a8887bafeff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:03.017227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd79872-8906-4a99-96eb-6a8887bafeff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:03.018706 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd79872-8906-4a99-96eb-6a8887bafeff None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:03.032241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 410/823] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:03.033158 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd79872-8906-4a99-96eb-6a8887bafeff None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:03.050441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c0ee0-2c49-4768-88e8-5fc93a26c43b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:03.089927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946c084-a762-4342-bb48-db0a31d5cd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:03.090564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946c084-a762-4342-bb48-db0a31d5cd45 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:03.090840 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946c084-a762-4342-bb48-db0a31d5cd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:03.091106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946c084-a762-4342-bb48-db0a31d5cd45 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:03.102002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946c084-a762-4342-bb48-db0a31d5cd45 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:03.103777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946c084-a762-4342-bb48-db0a31d5cd45 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-5946c084-a762-4342-bb48-db0a31d5cd45 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:03.115260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:03.121814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:03.123021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:03.123021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:03.123021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:03.123021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:03.123021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:03.123021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:03.123021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:03.123021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:03.123021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:24:03.123021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:03.123021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 414/824] 10.222.0.62 () {56 vars in 1117 bytes} [Thu Sep 1 16:24:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:03.123021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946c084-a762-4342-bb48-db0a31d5cd45 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:03.149122 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:03.152842 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd79872-8906-4a99-96eb-6a8887bafeff None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:03.174976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:03.192249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640c158-90a5-42d1-87f0-2705a98a8f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:03.193002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640c158-90a5-42d1-87f0-2705a98a8f07 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:03.193356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640c158-90a5-42d1-87f0-2705a98a8f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:03.193693 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640c158-90a5-42d1-87f0-2705a98a8f07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:03.194432 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640c158-90a5-42d1-87f0-2705a98a8f07 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:03.195818 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640c158-90a5-42d1-87f0-2705a98a8f07 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:03.220787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 411/825] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:24:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:24:03.221797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640c158-90a5-42d1-87f0-2705a98a8f07 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:03.248925 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946c084-a762-4342-bb48-db0a31d5cd45 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:03.302099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c13f57-0b9f-4b1d-9ea9-899ba30c14d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:03.303163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c13f57-0b9f-4b1d-9ea9-899ba30c14d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:03.303734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c13f57-0b9f-4b1d-9ea9-899ba30c14d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:03.304168 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c13f57-0b9f-4b1d-9ea9-899ba30c14d2 None admin] PATH_INFO: `/v3/projects/425048ffd13448cd8fe815333a9b798c/users/56f9bc7262b64bbab0f670576c6bdfdd/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:03.307344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c13f57-0b9f-4b1d-9ea9-899ba30c14d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=425048ffd13448cd8fe815333a9b798c, user_id=56f9bc7262b64bbab0f670576c6bdfdd, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:03.309323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c13f57-0b9f-4b1d-9ea9-899ba30c14d2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:03.328714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 415/826] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:24:03 2022] PUT /identity/v3/projects/425048ffd13448cd8fe815333a9b798c/users/56f9bc7262b64bbab0f670576c6bdfdd/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:24:03.374157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640c158-90a5-42d1-87f0-2705a98a8f07 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:03.419341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d967c-9325-4617-8a3b-467b68bd06e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:03.420226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d967c-9325-4617-8a3b-467b68bd06e8 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:03.420610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d967c-9325-4617-8a3b-467b68bd06e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:03.421027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d967c-9325-4617-8a3b-467b68bd06e8 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:03.421952 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d967c-9325-4617-8a3b-467b68bd06e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:03.422365 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d967c-9325-4617-8a3b-467b68bd06e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:03.424083 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d967c-9325-4617-8a3b-467b68bd06e8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:03.446091 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 412/827] 10.222.0.62 () {56 vars in 1149 bytes} [Thu Sep 1 16:24:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:03.451684 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c13f57-0b9f-4b1d-9ea9-899ba30c14d2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:03.569505 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c13f57-0b9f-4b1d-9ea9-899ba30c14d2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:03.577817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:03.608565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde85f6f-5aa5-42e6-8025-908ebf6be5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:03.609250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde85f6f-5aa5-42e6-8025-908ebf6be5cd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:03.609529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde85f6f-5aa5-42e6-8025-908ebf6be5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:03.609808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde85f6f-5aa5-42e6-8025-908ebf6be5cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:03.621070 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde85f6f-5aa5-42e6-8025-908ebf6be5cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:03.622550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde85f6f-5aa5-42e6-8025-908ebf6be5cd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-bde85f6f-5aa5-42e6-8025-908ebf6be5cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:03.638860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:03.640045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:03.641111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:03.641111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:03.641111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:03.644026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 416/828] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:03.646123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d967c-9325-4617-8a3b-467b68bd06e8 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:03.663151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:03.757252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d967c-9325-4617-8a3b-467b68bd06e8 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:03.794852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84039b-deaa-455b-8efb-66e786b02ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:03.795730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84039b-deaa-455b-8efb-66e786b02ffa None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:03.796166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84039b-deaa-455b-8efb-66e786b02ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:03.796564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84039b-deaa-455b-8efb-66e786b02ffa None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:03.797342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84039b-deaa-455b-8efb-66e786b02ffa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:03.797800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84039b-deaa-455b-8efb-66e786b02ffa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:03.799329 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84039b-deaa-455b-8efb-66e786b02ffa None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:03.816256 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 413/829] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:03.821765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde85f6f-5aa5-42e6-8025-908ebf6be5cd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:03.924269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde85f6f-5aa5-42e6-8025-908ebf6be5cd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:03.958717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308941e-ea89-45d1-9d83-d087376e7632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:03.959636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308941e-ea89-45d1-9d83-d087376e7632 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:03.960107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308941e-ea89-45d1-9d83-d087376e7632 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:03.960544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308941e-ea89-45d1-9d83-d087376e7632 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:03.967701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308941e-ea89-45d1-9d83-d087376e7632 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:03.969362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308941e-ea89-45d1-9d83-d087376e7632 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-6308941e-ea89-45d1-9d83-d087376e7632 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:03.980801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:03.981694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:03.982623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:03.982623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:03.982623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:03.982623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:03.982623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:03.982623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:03.982623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:03.982623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:03.984703 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 417/830] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:03.995956 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84039b-deaa-455b-8efb-66e786b02ffa None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:04.071738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:04.097159 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84039b-deaa-455b-8efb-66e786b02ffa None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:04.131434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d686c7e-8360-48a7-9e90-92417b0c1643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:04.132089 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d686c7e-8360-48a7-9e90-92417b0c1643 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:04.132371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d686c7e-8360-48a7-9e90-92417b0c1643 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:04.132634 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d686c7e-8360-48a7-9e90-92417b0c1643 None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:04.133317 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d686c7e-8360-48a7-9e90-92417b0c1643 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:04.133612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d686c7e-8360-48a7-9e90-92417b0c1643 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:04.135239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d686c7e-8360-48a7-9e90-92417b0c1643 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:04.147688 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 414/831] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:04.154079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308941e-ea89-45d1-9d83-d087376e7632 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:04.175273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:04.263856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308941e-ea89-45d1-9d83-d087376e7632 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:04.297954 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a01f12-488f-470c-9abd-fd49a8f60945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:04.298661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a01f12-488f-470c-9abd-fd49a8f60945 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:04.298934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a01f12-488f-470c-9abd-fd49a8f60945 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:04.299198 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a01f12-488f-470c-9abd-fd49a8f60945 None admin] PATH_INFO: `/v3/projects/425048ffd13448cd8fe815333a9b798c/users/24705faaf5054802a556749c7ba82ee8/roles/fc3db8095f31464ca4c378645d1d078b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:04.314548 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a01f12-488f-470c-9abd-fd49a8f60945 None admin] RBAC: Authorizing `identity:create_grant(project_id=425048ffd13448cd8fe815333a9b798c, user_id=24705faaf5054802a556749c7ba82ee8, role_id=fc3db8095f31464ca4c378645d1d078b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:04.316000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a01f12-488f-470c-9abd-fd49a8f60945 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:04.329664 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 418/832] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:24:04 2022] PUT /identity/v3/projects/425048ffd13448cd8fe815333a9b798c/users/24705faaf5054802a556749c7ba82ee8/roles/fc3db8095f31464ca4c378645d1d078b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:24:04.697052 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f86a1-f94a-4dcb-862f-9603501b16bf None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:04.697755 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f86a1-f94a-4dcb-862f-9603501b16bf None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:04.697755 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f86a1-f94a-4dcb-862f-9603501b16bf None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:04.699696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 415/833] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:04.705098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd051-57ad-4689-bd50-c5aa03f756d3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:04.705483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd051-57ad-4689-bd50-c5aa03f756d3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:04.705841 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd051-57ad-4689-bd50-c5aa03f756d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:04.724641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ad9bd051-57ad-4689-bd50-c5aa03f756d3 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:04.725949 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ad9bd051-57ad-4689-bd50-c5aa03f756d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:04.789723 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:04.800314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 419/834] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:05.067270 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f86a1-f94a-4dcb-862f-9603501b16bf None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:05.171284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f86a1-f94a-4dcb-862f-9603501b16bf None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:05.206275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427c647-9a25-411c-9b97-d7d04b426aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:05.206913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427c647-9a25-411c-9b97-d7d04b426aba None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:05.207188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427c647-9a25-411c-9b97-d7d04b426aba None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:05.207453 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427c647-9a25-411c-9b97-d7d04b426aba None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:05.208382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427c647-9a25-411c-9b97-d7d04b426aba None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:05.209818 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427c647-9a25-411c-9b97-d7d04b426aba None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:05.211666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-b427c647-9a25-411c-9b97-d7d04b426aba None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:05.219837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:05.221002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:05.222183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 416/835] 10.222.0.62 () {56 vars in 1132 bytes} [Thu Sep 1 16:24:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:05.228524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bd051-57ad-4689-bd50-c5aa03f756d3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:05.327134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9ee8b-8b85-40f0-bd82-72afc26da5a7 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:05.327344 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9ee8b-8b85-40f0-bd82-72afc26da5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:05.327572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9ee8b-8b85-40f0-bd82-72afc26da5a7 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:05.328432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bd051-57ad-4689-bd50-c5aa03f756d3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:05.329384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 417/836] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:05.334964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e811e-6d14-4575-8478-e44211733b7c None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:05.335252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e811e-6d14-4575-8478-e44211733b7c None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:05.335527 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e811e-6d14-4575-8478-e44211733b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:05.354979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-8a4e811e-6d14-4575-8478-e44211733b7c None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:05.356342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8a4e811e-6d14-4575-8478-e44211733b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:05.364328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e16083-34b0-404d-ba20-a8300cd20656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:05.364941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e16083-34b0-404d-ba20-a8300cd20656 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:05.365211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e16083-34b0-404d-ba20-a8300cd20656 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:05.365476 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e16083-34b0-404d-ba20-a8300cd20656 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:05.366100 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e16083-34b0-404d-ba20-a8300cd20656 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:05.366392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e16083-34b0-404d-ba20-a8300cd20656 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:05.367738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e16083-34b0-404d-ba20-a8300cd20656 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:05.372854 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 420/837] 10.222.0.62 () {56 vars in 1126 bytes} [Thu Sep 1 16:24:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:05.378304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e16083-34b0-404d-ba20-a8300cd20656 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:05.432302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 418/838] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:05.439654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:05.441400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c880e6-e6f2-4a59-accc-df642f48f18c None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:05.441693 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c880e6-e6f2-4a59-accc-df642f48f18c None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:05.442027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c880e6-e6f2-4a59-accc-df642f48f18c None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:05.448003 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 419/839] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:05 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:05.453426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d19f5e-ea07-4abc-9357-9cc91cfefe31 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:05.453711 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d19f5e-ea07-4abc-9357-9cc91cfefe31 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:05.453981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d19f5e-ea07-4abc-9357-9cc91cfefe31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:05.473667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-54d19f5e-ea07-4abc-9357-9cc91cfefe31 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:05.474901 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-54d19f5e-ea07-4abc-9357-9cc91cfefe31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:05.493677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e16083-34b0-404d-ba20-a8300cd20656 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:05.529604 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b75d4a-fe5f-4ba4-b219-fbd94d3c5fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:05.530674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b75d4a-fe5f-4ba4-b219-fbd94d3c5fac None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:05.531140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b75d4a-fe5f-4ba4-b219-fbd94d3c5fac None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:05.531593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b75d4a-fe5f-4ba4-b219-fbd94d3c5fac None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:05.532636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b75d4a-fe5f-4ba4-b219-fbd94d3c5fac None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:05.533125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b75d4a-fe5f-4ba4-b219-fbd94d3c5fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:05.534731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b75d4a-fe5f-4ba4-b219-fbd94d3c5fac None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:05.548455 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 421/840] 10.222.0.62 () {56 vars in 1126 bytes} [Thu Sep 1 16:24:05 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:05.560038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 420/841] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:05.706294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b75d4a-fe5f-4ba4-b219-fbd94d3c5fac None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:05.815253 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b75d4a-fe5f-4ba4-b219-fbd94d3c5fac None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:05.833523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d19f5e-ea07-4abc-9357-9cc91cfefe31 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:05.842973 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:05.854447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a432c43-1f08-4e47-a9d6-92c05f8fa323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:05.855642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a432c43-1f08-4e47-a9d6-92c05f8fa323 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:05.856149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a432c43-1f08-4e47-a9d6-92c05f8fa323 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:05.857090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a432c43-1f08-4e47-a9d6-92c05f8fa323 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:05.858150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a432c43-1f08-4e47-a9d6-92c05f8fa323 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:05.860355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a432c43-1f08-4e47-a9d6-92c05f8fa323 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-5a432c43-1f08-4e47-a9d6-92c05f8fa323 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:05.871318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:05.872383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:05.873212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 422/842] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:24:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:05.878388 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a432c43-1f08-4e47-a9d6-92c05f8fa323 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:05.915609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:05.967578 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d19f5e-ea07-4abc-9357-9cc91cfefe31 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:06.000270 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a432c43-1f08-4e47-a9d6-92c05f8fa323 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:06.014915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df131841-17fd-426a-8b7c-48201dbad0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:06.016324 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df131841-17fd-426a-8b7c-48201dbad0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:06.016607 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df131841-17fd-426a-8b7c-48201dbad0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:06.017680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df131841-17fd-426a-8b7c-48201dbad0c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:06.031672 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df131841-17fd-426a-8b7c-48201dbad0c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-df131841-17fd-426a-8b7c-48201dbad0c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:06.037775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:06.038690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:06.039524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:06.039524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:06.039524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:06.039524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:06.039524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:06.039524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:06.039524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:06.039524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:06.039524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:06.039524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:06.039524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:06.039524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:06.040177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22bc3f-91fe-4bfd-8a0b-70638b6578a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:06.040177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22bc3f-91fe-4bfd-8a0b-70638b6578a5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:06.040905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22bc3f-91fe-4bfd-8a0b-70638b6578a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:06.041002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22bc3f-91fe-4bfd-8a0b-70638b6578a5 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:06.041568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22bc3f-91fe-4bfd-8a0b-70638b6578a5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:06.041867 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22bc3f-91fe-4bfd-8a0b-70638b6578a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:06.042825 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 421/843] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:24:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:06.043224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22bc3f-91fe-4bfd-8a0b-70638b6578a5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:06.048666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df131841-17fd-426a-8b7c-48201dbad0c6 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:06.052670 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 423/844] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:24:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:06.053537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22bc3f-91fe-4bfd-8a0b-70638b6578a5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:06.117906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:06.170558 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df131841-17fd-426a-8b7c-48201dbad0c6 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:06.178025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22bc3f-91fe-4bfd-8a0b-70638b6578a5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:06.190743 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:06.210652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381f951-ac2e-4f02-a2b9-922dc6a017b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:06.211271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381f951-ac2e-4f02-a2b9-922dc6a017b8 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:06.211548 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381f951-ac2e-4f02-a2b9-922dc6a017b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:06.211815 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381f951-ac2e-4f02-a2b9-922dc6a017b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:06.212750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381f951-ac2e-4f02-a2b9-922dc6a017b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:06.213084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381f951-ac2e-4f02-a2b9-922dc6a017b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:06.214529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381f951-ac2e-4f02-a2b9-922dc6a017b8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:06.223954 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 422/845] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:24:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:06.225549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298bd95-4eda-4cfb-bbd8-b3dfddcca586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:06.226194 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298bd95-4eda-4cfb-bbd8-b3dfddcca586 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:06.226472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298bd95-4eda-4cfb-bbd8-b3dfddcca586 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:06.226740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298bd95-4eda-4cfb-bbd8-b3dfddcca586 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:06.227372 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298bd95-4eda-4cfb-bbd8-b3dfddcca586 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:06.227670 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298bd95-4eda-4cfb-bbd8-b3dfddcca586 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:06.229194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381f951-ac2e-4f02-a2b9-922dc6a017b8 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:06.230596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298bd95-4eda-4cfb-bbd8-b3dfddcca586 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:06.242525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 424/846] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:24:06 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:06.350375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381f951-ac2e-4f02-a2b9-922dc6a017b8 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:06.350618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c2ec9-e778-4f12-b76b-f6e208eb2346 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:06.350704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c2ec9-e778-4f12-b76b-f6e208eb2346 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:06.350951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c2ec9-e778-4f12-b76b-f6e208eb2346 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:06.354796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 425/847] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:06.358883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00c8e4-b7be-437c-8126-c46328bb685f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:06.359162 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00c8e4-b7be-437c-8126-c46328bb685f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:06.359426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00c8e4-b7be-437c-8126-c46328bb685f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:06.379276 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7b00c8e4-b7be-437c-8126-c46328bb685f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:06.380552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7b00c8e4-b7be-437c-8126-c46328bb685f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:06.395416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b07683-2d4e-4c90-92fd-db27a9ac3569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:06.396116 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b07683-2d4e-4c90-92fd-db27a9ac3569 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:06.396389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b07683-2d4e-4c90-92fd-db27a9ac3569 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:06.396651 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b07683-2d4e-4c90-92fd-db27a9ac3569 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:06.408408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b07683-2d4e-4c90-92fd-db27a9ac3569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:06.409875 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b07683-2d4e-4c90-92fd-db27a9ac3569 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-f2b07683-2d4e-4c90-92fd-db27a9ac3569 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:06.426268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:06.427244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:06.428184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:06.428184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:06.428184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:06.428184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 423/848] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:06.434228 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b07683-2d4e-4c90-92fd-db27a9ac3569 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:06.472472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 426/849] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:06.473584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:06.538882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b07683-2d4e-4c90-92fd-db27a9ac3569 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:06.572864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefd40c-af21-4be1-89c8-fb303f2c5e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:06.573474 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefd40c-af21-4be1-89c8-fb303f2c5e13 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:06.573755 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefd40c-af21-4be1-89c8-fb303f2c5e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:06.574007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefd40c-af21-4be1-89c8-fb303f2c5e13 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:06.574642 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefd40c-af21-4be1-89c8-fb303f2c5e13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:06.574937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefd40c-af21-4be1-89c8-fb303f2c5e13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:06.576342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefd40c-af21-4be1-89c8-fb303f2c5e13 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:06.589475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 424/850] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:06.595136 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00c8e4-b7be-437c-8126-c46328bb685f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:06.647310 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:06.702314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00c8e4-b7be-437c-8126-c46328bb685f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:06.738941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf536f-33b8-46ba-a49c-599899bf1118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:06.739578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf536f-33b8-46ba-a49c-599899bf1118 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:06.739852 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf536f-33b8-46ba-a49c-599899bf1118 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:06.740162 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf536f-33b8-46ba-a49c-599899bf1118 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:06.747643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf536f-33b8-46ba-a49c-599899bf1118 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:06.749140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf536f-33b8-46ba-a49c-599899bf1118 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:06.754334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefd40c-af21-4be1-89c8-fb303f2c5e13 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a3bf536f-33b8-46ba-a49c-599899bf1118 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:06.777314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:06.778302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:06.779856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:06.779856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:06.779856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:06.779856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:06.779856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:06.779856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:06.779856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:06.779856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:06.779856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:24:06.779856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:06.779856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 427/851] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:24:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:06.784557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf536f-33b8-46ba-a49c-599899bf1118 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:06.862161 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:06.872522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefd40c-af21-4be1-89c8-fb303f2c5e13 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:06.904166 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf536f-33b8-46ba-a49c-599899bf1118 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:06.909629 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80362f3-8800-4469-97fb-07c14b89196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:06.910392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80362f3-8800-4469-97fb-07c14b89196a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:06.910763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80362f3-8800-4469-97fb-07c14b89196a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:06.911116 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80362f3-8800-4469-97fb-07c14b89196a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:06.914949 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:06.924567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80362f3-8800-4469-97fb-07c14b89196a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-a80362f3-8800-4469-97fb-07c14b89196a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:06.930801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:06.932269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:06.933293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:06.933293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:06.933293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:06.933293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:06.933293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:06.933293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:06.933293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:06.933293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:06.933293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:06.933293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:06.933293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:24:06.933293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:06.940602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 425/852] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:24:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:06.941984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80362f3-8800-4469-97fb-07c14b89196a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:06.947767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d527f-7e42-429f-be45-040e34db43ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:06.948546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d527f-7e42-429f-be45-040e34db43ba None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:06.948913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d527f-7e42-429f-be45-040e34db43ba None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:06.949275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d527f-7e42-429f-be45-040e34db43ba None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:06.950103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d527f-7e42-429f-be45-040e34db43ba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:06.950494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d527f-7e42-429f-be45-040e34db43ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:06.952048 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d527f-7e42-429f-be45-040e34db43ba None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:06.977419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 428/853] 10.222.0.62 () {56 vars in 1135 bytes} [Thu Sep 1 16:24:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:06.982413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d527f-7e42-429f-be45-040e34db43ba None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:07.053753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80362f3-8800-4469-97fb-07c14b89196a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:07.090005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d527f-7e42-429f-be45-040e34db43ba None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:07.091221 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e7c24-729b-4067-961c-2ffb95ddb155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:07.092185 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e7c24-729b-4067-961c-2ffb95ddb155 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:07.092730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e7c24-729b-4067-961c-2ffb95ddb155 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:07.093223 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e7c24-729b-4067-961c-2ffb95ddb155 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:07.094077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e7c24-729b-4067-961c-2ffb95ddb155 None admin] RBAC: Adding query filter params (name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:07.094570 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e7c24-729b-4067-961c-2ffb95ddb155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:07.096157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e7c24-729b-4067-961c-2ffb95ddb155 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:07.110380 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 426/854] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:24:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:07.112012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e7c24-729b-4067-961c-2ffb95ddb155 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:07.126567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94c1ed-ad82-4b67-89e5-6b942fa13ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:07.127154 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94c1ed-ad82-4b67-89e5-6b942fa13ddf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:07.127429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94c1ed-ad82-4b67-89e5-6b942fa13ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:07.127696 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94c1ed-ad82-4b67-89e5-6b942fa13ddf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:07.131104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:07.162744 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:07.198292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94c1ed-ad82-4b67-89e5-6b942fa13ddf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:07.199738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94c1ed-ad82-4b67-89e5-6b942fa13ddf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:07.229933 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e7c24-729b-4067-961c-2ffb95ddb155 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:07.267918 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27af3ca-7d88-43de-bb7e-b39aec945445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:07.268904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27af3ca-7d88-43de-bb7e-b39aec945445 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:07.269375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27af3ca-7d88-43de-bb7e-b39aec945445 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:07.269945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27af3ca-7d88-43de-bb7e-b39aec945445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:07.281940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27af3ca-7d88-43de-bb7e-b39aec945445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:07.283803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27af3ca-7d88-43de-bb7e-b39aec945445 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-e27af3ca-7d88-43de-bb7e-b39aec945445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:07.300418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:07.301437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:07.302367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:07.302367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:07.302367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:07.309386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 429/856] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:06 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:24:07.309386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94c1ed-ad82-4b67-89e5-6b942fa13ddf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:07.310153 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 427/856] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:07.311606 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27af3ca-7d88-43de-bb7e-b39aec945445 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:07.378681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:07.389204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:07.414828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94c1ed-ad82-4b67-89e5-6b942fa13ddf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:07.432714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27af3ca-7d88-43de-bb7e-b39aec945445 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:07.452520 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf1d2a-d722-4810-a4d2-ef303a09f5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:07.453072 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf1d2a-d722-4810-a4d2-ef303a09f5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:07.453389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf1d2a-d722-4810-a4d2-ef303a09f5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:07.453659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf1d2a-d722-4810-a4d2-ef303a09f5c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:07.464483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf1d2a-d722-4810-a4d2-ef303a09f5c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:07.465924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf1d2a-d722-4810-a4d2-ef303a09f5c4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:07.472244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ceef7-532c-4249-8510-9bbfcf41c0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:07.473193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ceef7-532c-4249-8510-9bbfcf41c0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:07.473687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ceef7-532c-4249-8510-9bbfcf41c0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:07.474147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ceef7-532c-4249-8510-9bbfcf41c0e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:07.475092 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ceef7-532c-4249-8510-9bbfcf41c0e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:07.475584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ceef7-532c-4249-8510-9bbfcf41c0e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:07.477161 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ceef7-532c-4249-8510-9bbfcf41c0e8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9ecf1d2a-d722-4810-a4d2-ef303a09f5c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:07.481468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:07.482401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:07.483286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:07.483286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:07.483286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:07.487276 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 430/857] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:07.488422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf1d2a-d722-4810-a4d2-ef303a09f5c4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:07.498572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 428/858] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:07.500241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ceef7-532c-4249-8510-9bbfcf41c0e8 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:07.593778 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf1d2a-d722-4810-a4d2-ef303a09f5c4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:07.605516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ceef7-532c-4249-8510-9bbfcf41c0e8 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:07.612920 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:07.631231 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:07.633409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078e2ac-25fc-4305-9f13-238eced737c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:07.634271 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078e2ac-25fc-4305-9f13-238eced737c7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:07.634742 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078e2ac-25fc-4305-9f13-238eced737c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:07.635124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078e2ac-25fc-4305-9f13-238eced737c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:07.635772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078e2ac-25fc-4305-9f13-238eced737c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:07.636103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078e2ac-25fc-4305-9f13-238eced737c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:07.637482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078e2ac-25fc-4305-9f13-238eced737c7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:07.641886 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754decb-e370-4f1a-947f-b953300ca262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:07.642807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754decb-e370-4f1a-947f-b953300ca262 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:07.643257 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754decb-e370-4f1a-947f-b953300ca262 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:07.643718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754decb-e370-4f1a-947f-b953300ca262 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:07.649504 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 431/859] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:07.650989 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf147a-0ae3-4585-8c41-7b31255272ee None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:07.651280 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf147a-0ae3-4585-8c41-7b31255272ee None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:07.651553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf147a-0ae3-4585-8c41-7b31255272ee None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:07.652959 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 432/860] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:07.655769 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf147a-0ae3-4585-8c41-7b31255272ee None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:07.657154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4754decb-e370-4f1a-947f-b953300ca262 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:07.658631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4754decb-e370-4f1a-947f-b953300ca262 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-4754decb-e370-4f1a-947f-b953300ca262 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:07.671302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:07.672319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:07.673287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:07.673287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:07.673287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:07.673287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:07.673287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:07.673287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:07.673287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:07.673287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:07.673287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:24:07.673287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:07.673287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 429/861] 10.222.0.62 () {56 vars in 1117 bytes} [Thu Sep 1 16:24:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:07.673889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07882ead-afda-4541-b06d-06ee7684abdd None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:07.673889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07882ead-afda-4541-b06d-06ee7684abdd None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:07.674952 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07882ead-afda-4541-b06d-06ee7684abdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:07.692425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-07882ead-afda-4541-b06d-06ee7684abdd None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:07.693611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-07882ead-afda-4541-b06d-06ee7684abdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:07.779927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 430/862] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:07.779927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07882ead-afda-4541-b06d-06ee7684abdd None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:07.780948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf147a-0ae3-4585-8c41-7b31255272ee None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:07.827001 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228d6d3-f494-47c2-b69b-eb175622b979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:07.827643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228d6d3-f494-47c2-b69b-eb175622b979 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:07.827938 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228d6d3-f494-47c2-b69b-eb175622b979 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:07.828210 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228d6d3-f494-47c2-b69b-eb175622b979 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:07.838059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228d6d3-f494-47c2-b69b-eb175622b979 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:07.840187 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228d6d3-f494-47c2-b69b-eb175622b979 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-4228d6d3-f494-47c2-b69b-eb175622b979 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:07.855182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:07.856163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:07.857062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:07.857062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:07.857062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:07.857062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:07.857062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:07.857062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:07.857062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:07.857062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:07.860031 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 433/863] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:07.863370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228d6d3-f494-47c2-b69b-eb175622b979 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:07.887078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07882ead-afda-4541-b06d-06ee7684abdd None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:07.904732 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:07.919936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:07.924882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066e18f-a67e-466e-9ad9-44fe701f4f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:07.925757 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066e18f-a67e-466e-9ad9-44fe701f4f78 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:07.926040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066e18f-a67e-466e-9ad9-44fe701f4f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:07.926299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066e18f-a67e-466e-9ad9-44fe701f4f78 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:07.927208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066e18f-a67e-466e-9ad9-44fe701f4f78 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:07.927508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066e18f-a67e-466e-9ad9-44fe701f4f78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:07.929059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066e18f-a67e-466e-9ad9-44fe701f4f78 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:07.952698 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 431/864] 10.222.0.62 () {56 vars in 1149 bytes} [Thu Sep 1 16:24:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:07.954248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066e18f-a67e-466e-9ad9-44fe701f4f78 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:07.993500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228d6d3-f494-47c2-b69b-eb175622b979 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:08.034494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8bfe9-4022-43e8-b33a-e592217f6e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:08.035867 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8bfe9-4022-43e8-b33a-e592217f6e5b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:08.036373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8bfe9-4022-43e8-b33a-e592217f6e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:08.037091 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8bfe9-4022-43e8-b33a-e592217f6e5b None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:08.038005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8bfe9-4022-43e8-b33a-e592217f6e5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:08.038691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8bfe9-4022-43e8-b33a-e592217f6e5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:08.041399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8bfe9-4022-43e8-b33a-e592217f6e5b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:08.062357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 434/865] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:08.063729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8bfe9-4022-43e8-b33a-e592217f6e5b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:08.065673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066e18f-a67e-466e-9ad9-44fe701f4f78 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:08.104462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b1616-80ea-4348-bb28-bfe9afc1ca88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:08.105079 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1616-80ea-4348-bb28-bfe9afc1ca88 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:08.105353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1616-80ea-4348-bb28-bfe9afc1ca88 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:08.105648 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b1616-80ea-4348-bb28-bfe9afc1ca88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:08.151082 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:08.174374 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:08.178803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8bfe9-4022-43e8-b33a-e592217f6e5b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:08.179040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b1616-80ea-4348-bb28-bfe9afc1ca88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:08.180550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b1616-80ea-4348-bb28-bfe9afc1ca88 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:08.217580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6e506-f38e-4823-94c5-433931fc33d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:08.218287 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6e506-f38e-4823-94c5-433931fc33d8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:08.218565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6e506-f38e-4823-94c5-433931fc33d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:08.218832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6e506-f38e-4823-94c5-433931fc33d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:08.219668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6e506-f38e-4823-94c5-433931fc33d8 None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=56f9bc7262b64bbab0f670576c6bdfdd) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:08.220008 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6e506-f38e-4823-94c5-433931fc33d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:08.221431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6e506-f38e-4823-94c5-433931fc33d8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:08.235927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 435/866] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:08 2022] GET /identity/v3/role_assignments?user.id=56f9bc7262b64bbab0f670576c6bdfdd&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:08.235927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6e506-f38e-4823-94c5-433931fc33d8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:08.303210 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 432/867] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:07 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:24:08.304667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef4656-614f-4503-a62c-ace97d0eb203 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:08.305007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef4656-614f-4503-a62c-ace97d0eb203 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:08.305339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef4656-614f-4503-a62c-ace97d0eb203 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:08.311109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 433/868] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:08.312073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef4656-614f-4503-a62c-ace97d0eb203 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:08.355214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6e506-f38e-4823-94c5-433931fc33d8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:08.398081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881fbca-5602-41e5-83e1-5716a52353fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:08.398770 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881fbca-5602-41e5-83e1-5716a52353fb None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:08.399049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881fbca-5602-41e5-83e1-5716a52353fb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:08.400097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881fbca-5602-41e5-83e1-5716a52353fb None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:08.400097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881fbca-5602-41e5-83e1-5716a52353fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:08.401154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:08.402141 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881fbca-5602-41e5-83e1-5716a52353fb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:08.422087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 436/869] 10.222.0.62 () {60 vars in 1141 bytes} [Thu Sep 1 16:24:08 2022] POST /identity/v3/services => generated 269 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:24:08.423410 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb9067-532c-4268-b6ca-3734b8c0ad30 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:08.423843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb9067-532c-4268-b6ca-3734b8c0ad30 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:08.424150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb9067-532c-4268-b6ca-3734b8c0ad30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:08.445047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-49fb9067-532c-4268-b6ca-3734b8c0ad30 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:08.445870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-49fb9067-532c-4268-b6ca-3734b8c0ad30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:08.452376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef4656-614f-4503-a62c-ace97d0eb203 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:08.469323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:08.494974 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59a7c5-cd10-498b-ab15-0ae34f9dadbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:08.495772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59a7c5-cd10-498b-ab15-0ae34f9dadbb None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:08.496096 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59a7c5-cd10-498b-ab15-0ae34f9dadbb None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:08.496421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59a7c5-cd10-498b-ab15-0ae34f9dadbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:08.510109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59a7c5-cd10-498b-ab15-0ae34f9dadbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:08.511480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59a7c5-cd10-498b-ab15-0ae34f9dadbb None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-2b59a7c5-cd10-498b-ab15-0ae34f9dadbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:08.529274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:08.530277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:08.531184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:08.531184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:08.531184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:08.535352 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 434/870] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:08.536820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59a7c5-cd10-498b-ab15-0ae34f9dadbb None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:08.543637 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 437/871] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:08 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:08.650341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59a7c5-cd10-498b-ab15-0ae34f9dadbb None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:08.677326 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:08.686628 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e624c8-f237-4224-9203-987da3bdf950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:08.687246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e624c8-f237-4224-9203-987da3bdf950 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:08.687524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e624c8-f237-4224-9203-987da3bdf950 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:08.687791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e624c8-f237-4224-9203-987da3bdf950 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:08.688475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e624c8-f237-4224-9203-987da3bdf950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:08.688775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e624c8-f237-4224-9203-987da3bdf950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:08.690150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e624c8-f237-4224-9203-987da3bdf950 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:08.707916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 435/872] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:08.709278 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb9067-532c-4268-b6ca-3734b8c0ad30 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:08.817135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb9067-532c-4268-b6ca-3734b8c0ad30 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:08.818465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e624c8-f237-4224-9203-987da3bdf950 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:08.859076 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5226a-3363-420b-8635-6f23a6a8c5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:08.859711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5226a-3363-420b-8635-6f23a6a8c5b5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:08.860037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5226a-3363-420b-8635-6f23a6a8c5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:08.860317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5226a-3363-420b-8635-6f23a6a8c5b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:08.867114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5226a-3363-420b-8635-6f23a6a8c5b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:08.868621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5226a-3363-420b-8635-6f23a6a8c5b5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-65f5226a-3363-420b-8635-6f23a6a8c5b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:08.880513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:08.881440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:08.882361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:08.882361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:08.882361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:08.882361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:08.882361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:08.882361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:08.882361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:08.882361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:08.887327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 438/873] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:08.888265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5226a-3363-420b-8635-6f23a6a8c5b5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:08.904557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:08.956626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e624c8-f237-4224-9203-987da3bdf950 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:09.001886 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec943349-d235-404c-ab44-1e6f3c102fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:09.002845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec943349-d235-404c-ab44-1e6f3c102fc4 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:09.003123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec943349-d235-404c-ab44-1e6f3c102fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:09.003390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec943349-d235-404c-ab44-1e6f3c102fc4 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:09.004072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec943349-d235-404c-ab44-1e6f3c102fc4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:09.005449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec943349-d235-404c-ab44-1e6f3c102fc4 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:09.005867 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5226a-3363-420b-8635-6f23a6a8c5b5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:09.028114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 436/874] 10.222.0.62 () {60 vars in 1141 bytes} [Thu Sep 1 16:24:08 2022] POST /identity/v3/services => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:24:09.061890 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64479a-152e-44c7-9817-52c56f319b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:09.062937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64479a-152e-44c7-9817-52c56f319b33 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:09.063460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64479a-152e-44c7-9817-52c56f319b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:09.064027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64479a-152e-44c7-9817-52c56f319b33 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:09.065040 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64479a-152e-44c7-9817-52c56f319b33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:09.065548 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64479a-152e-44c7-9817-52c56f319b33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:09.067208 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64479a-152e-44c7-9817-52c56f319b33 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:09.082960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 439/875] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:09.090215 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec943349-d235-404c-ab44-1e6f3c102fc4 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:09.148424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:09.215339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec943349-d235-404c-ab44-1e6f3c102fc4 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:09.252045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f74544-214f-4ffa-b35b-e13fc4fd919b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:09.252954 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f74544-214f-4ffa-b35b-e13fc4fd919b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:09.253372 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f74544-214f-4ffa-b35b-e13fc4fd919b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:09.253843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f74544-214f-4ffa-b35b-e13fc4fd919b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:09.254915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f74544-214f-4ffa-b35b-e13fc4fd919b None admin] RBAC: Adding query filter params (role.id=fc3db8095f31464ca4c378645d1d078b, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=24705faaf5054802a556749c7ba82ee8) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:09.255364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f74544-214f-4ffa-b35b-e13fc4fd919b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:09.257169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f74544-214f-4ffa-b35b-e13fc4fd919b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:09.264546 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 437/876] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:09 2022] GET /identity/v3/role_assignments?user.id=24705faaf5054802a556749c7ba82ee8&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=fc3db8095f31464ca4c378645d1d078b => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:10.208041 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69e360-4811-493a-8b00-81ca78484bf5 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:10.208572 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69e360-4811-493a-8b00-81ca78484bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:10.208572 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69e360-4811-493a-8b00-81ca78484bf5 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:10.210123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 440/877] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:10.216099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53840b1-917d-4b80-a4f4-d45771760da4 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:10.216510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53840b1-917d-4b80-a4f4-d45771760da4 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:10.216685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53840b1-917d-4b80-a4f4-d45771760da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:10.235225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-c53840b1-917d-4b80-a4f4-d45771760da4 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:10.236458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c53840b1-917d-4b80-a4f4-d45771760da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:10.309441 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 438/878] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:10.453317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e6a81-848e-45cc-ae94-08ae429fc891 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:10.453657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e6a81-848e-45cc-ae94-08ae429fc891 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:10.453863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e6a81-848e-45cc-ae94-08ae429fc891 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:10.455469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 441/879] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:10.460770 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e668ef1-cd66-4d0f-b669-8e5f71e29d7a None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:10.460992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e668ef1-cd66-4d0f-b669-8e5f71e29d7a None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:10.461260 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e668ef1-cd66-4d0f-b669-8e5f71e29d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:10.480176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-5e668ef1-cd66-4d0f-b669-8e5f71e29d7a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:10.481343 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5e668ef1-cd66-4d0f-b669-8e5f71e29d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:10.511117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:10.553480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 439/880] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:10.580449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e6a81-848e-45cc-ae94-08ae429fc891 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:10.654113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:10.691238 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e6a81-848e-45cc-ae94-08ae429fc891 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:10.731927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897817e3-a3b7-4946-b374-183e6f56d116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:10.732487 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897817e3-a3b7-4946-b374-183e6f56d116 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:10.732758 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897817e3-a3b7-4946-b374-183e6f56d116 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:10.733023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897817e3-a3b7-4946-b374-183e6f56d116 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:10.733610 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897817e3-a3b7-4946-b374-183e6f56d116 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:10.734962 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897817e3-a3b7-4946-b374-183e6f56d116 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-897817e3-a3b7-4946-b374-183e6f56d116 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:10.744739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:10.745706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:10.746526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 442/881] 10.222.0.62 () {56 vars in 1126 bytes} [Thu Sep 1 16:24:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:10.752230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e668ef1-cd66-4d0f-b669-8e5f71e29d7a None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:10.823528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897817e3-a3b7-4946-b374-183e6f56d116 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:10.871299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e668ef1-cd66-4d0f-b669-8e5f71e29d7a None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:10.909611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cc27d-1291-4440-8611-5a12b7acff60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:10.910600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cc27d-1291-4440-8611-5a12b7acff60 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:10.911030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cc27d-1291-4440-8611-5a12b7acff60 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:10.911434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cc27d-1291-4440-8611-5a12b7acff60 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:10.912244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cc27d-1291-4440-8611-5a12b7acff60 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:10.912677 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cc27d-1291-4440-8611-5a12b7acff60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:10.914232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cc27d-1291-4440-8611-5a12b7acff60 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:10.920323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 440/882] 10.222.0.62 () {56 vars in 1122 bytes} [Thu Sep 1 16:24:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:10.930596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cc27d-1291-4440-8611-5a12b7acff60 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:10.943685 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897817e3-a3b7-4946-b374-183e6f56d116 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:10.964308 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:10.979459 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:10.986740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba37313-0641-492a-9630-1946ca791e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:10.987583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba37313-0641-492a-9630-1946ca791e1a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:10.988123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba37313-0641-492a-9630-1946ca791e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:10.988538 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba37313-0641-492a-9630-1946ca791e1a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:10.989362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba37313-0641-492a-9630-1946ca791e1a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:10.990951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba37313-0641-492a-9630-1946ca791e1a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-0ba37313-0641-492a-9630-1946ca791e1a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:11.000851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:11.001984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:11.002885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:11.002885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:11.002885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:11.002885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:11.002885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:11.002885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:11.002885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:11.002885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:11.002885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:11.002885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:11.002885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:11.002885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:11.002885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:24:11.002885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:11.005456 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 443/883] 10.222.0.62 () {56 vars in 1132 bytes} [Thu Sep 1 16:24:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:11.017343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba37313-0641-492a-9630-1946ca791e1a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:11.043554 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cc27d-1291-4440-8611-5a12b7acff60 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:11.085249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1c07c-58bb-4bb3-9aec-0e474c8bdb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:11.085862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1c07c-58bb-4bb3-9aec-0e474c8bdb45 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.086132 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1c07c-58bb-4bb3-9aec-0e474c8bdb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.086637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1c07c-58bb-4bb3-9aec-0e474c8bdb45 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.087820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1c07c-58bb-4bb3-9aec-0e474c8bdb45 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:11.088164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1c07c-58bb-4bb3-9aec-0e474c8bdb45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:11.090047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1c07c-58bb-4bb3-9aec-0e474c8bdb45 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:11.096902 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 441/884] 10.222.0.62 () {56 vars in 1122 bytes} [Thu Sep 1 16:24:10 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:11.124066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69b1a8-f72d-4752-b22c-2b161bd98c63 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.124582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69b1a8-f72d-4752-b22c-2b161bd98c63 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.124994 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69b1a8-f72d-4752-b22c-2b161bd98c63 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.131088 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 442/885] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:11.133275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8d6a9-ce53-4cff-9fd4-b7c2d3e7abce None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.133772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8d6a9-ce53-4cff-9fd4-b7c2d3e7abce None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.134561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8d6a9-ce53-4cff-9fd4-b7c2d3e7abce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.144271 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba37313-0641-492a-9630-1946ca791e1a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:11.156276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-51f8d6a9-ce53-4cff-9fd4-b7c2d3e7abce None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:11.157857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-51f8d6a9-ce53-4cff-9fd4-b7c2d3e7abce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:11.182951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f8fc1-60ab-416d-8e48-b99ae533a084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:11.183575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f8fc1-60ab-416d-8e48-b99ae533a084 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.183852 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f8fc1-60ab-416d-8e48-b99ae533a084 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.184157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f8fc1-60ab-416d-8e48-b99ae533a084 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.184986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f8fc1-60ab-416d-8e48-b99ae533a084 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:11.185289 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f8fc1-60ab-416d-8e48-b99ae533a084 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:11.186752 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f8fc1-60ab-416d-8e48-b99ae533a084 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:11.196921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 444/886] 10.222.0.62 () {56 vars in 1126 bytes} [Thu Sep 1 16:24:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:11.199436 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f8fc1-60ab-416d-8e48-b99ae533a084 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:11.252011 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 443/887] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:11.276462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:11.291854 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dcfa1-df5c-41de-9156-be16e8d6d5fd None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.292506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dcfa1-df5c-41de-9156-be16e8d6d5fd None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.293003 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dcfa1-df5c-41de-9156-be16e8d6d5fd None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.294631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 444/888] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:11.299635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2f332-c4ab-4a46-a9b7-3dcfd9fc8eff None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.299971 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2f332-c4ab-4a46-a9b7-3dcfd9fc8eff None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.300213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2f332-c4ab-4a46-a9b7-3dcfd9fc8eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.327914 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-0ec2f332-c4ab-4a46-a9b7-3dcfd9fc8eff None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:11.327914 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0ec2f332-c4ab-4a46-a9b7-3dcfd9fc8eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:11.331411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f8fc1-60ab-416d-8e48-b99ae533a084 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:11.349027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:11.370040 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48fdda-ef9d-44e3-be5c-23ad4b6874e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:11.370799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48fdda-ef9d-44e3-be5c-23ad4b6874e8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.371072 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48fdda-ef9d-44e3-be5c-23ad4b6874e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.371335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48fdda-ef9d-44e3-be5c-23ad4b6874e8 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.371985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48fdda-ef9d-44e3-be5c-23ad4b6874e8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:11.372281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48fdda-ef9d-44e3-be5c-23ad4b6874e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:11.373786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48fdda-ef9d-44e3-be5c-23ad4b6874e8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:11.379324 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 445/889] 10.222.0.62 () {56 vars in 1126 bytes} [Thu Sep 1 16:24:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:11.389232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48fdda-ef9d-44e3-be5c-23ad4b6874e8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:11.419980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 445/890] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:11.501343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48fdda-ef9d-44e3-be5c-23ad4b6874e8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:11.525691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:11.528752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2f332-c4ab-4a46-a9b7-3dcfd9fc8eff None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:11.539068 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5af4db-377d-4cf4-838c-68933ae311fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:11.540162 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5af4db-377d-4cf4-838c-68933ae311fa None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.540471 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5af4db-377d-4cf4-838c-68933ae311fa None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.540751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5af4db-377d-4cf4-838c-68933ae311fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.542464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5af4db-377d-4cf4-838c-68933ae311fa None admin] RBAC: Adding query filter params (interface=public, service_id=9d81094ebb5d414193a4cb2233beb13b, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:11.542896 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5af4db-377d-4cf4-838c-68933ae311fa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:11.544542 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5af4db-377d-4cf4-838c-68933ae311fa None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:11.555998 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 446/891] 10.222.0.62 () {56 vars in 1251 bytes} [Thu Sep 1 16:24:11 2022] GET /identity/v3/endpoints?service_id=9d81094ebb5d414193a4cb2233beb13b&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:11.561580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5af4db-377d-4cf4-838c-68933ae311fa None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:11.642222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2f332-c4ab-4a46-a9b7-3dcfd9fc8eff None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:11.684327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f1b97-3cc2-4213-84fd-80f9e569b7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:11.685256 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f1b97-3cc2-4213-84fd-80f9e569b7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.685546 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f1b97-3cc2-4213-84fd-80f9e569b7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.685817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f1b97-3cc2-4213-84fd-80f9e569b7a7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.686555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f1b97-3cc2-4213-84fd-80f9e569b7a7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:11.688084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f1b97-3cc2-4213-84fd-80f9e569b7a7 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:11.689034 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5af4db-377d-4cf4-838c-68933ae311fa None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-500f1b97-3cc2-4213-84fd-80f9e569b7a7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:11.698630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:11.699665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:11.701160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 446/892] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:24:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:11.705690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f1b97-3cc2-4213-84fd-80f9e569b7a7 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:11.725972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:11.733901 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d787d2-af23-4882-988d-1539e8bd7ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:11.734522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d787d2-af23-4882-988d-1539e8bd7ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.734794 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d787d2-af23-4882-988d-1539e8bd7ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.735059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d787d2-af23-4882-988d-1539e8bd7ac0 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.735695 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d787d2-af23-4882-988d-1539e8bd7ac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:11.737293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d787d2-af23-4882-988d-1539e8bd7ac0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:11.743726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 447/893] 10.222.0.62 () {56 vars in 1087 bytes} [Thu Sep 1 16:24:11 2022] GET /identity/v3/services => generated 1268 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:24:11.744409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d787d2-af23-4882-988d-1539e8bd7ac0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:11.816021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:11.837145 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f1b97-3cc2-4213-84fd-80f9e569b7a7 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:11.876587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d787d2-af23-4882-988d-1539e8bd7ac0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:11.888763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5e91d-426e-469e-b521-7ba71eb8f84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:11.889429 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5e91d-426e-469e-b521-7ba71eb8f84e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.890122 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5e91d-426e-469e-b521-7ba71eb8f84e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.890404 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5e91d-426e-469e-b521-7ba71eb8f84e None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.891071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5e91d-426e-469e-b521-7ba71eb8f84e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:11.891693 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5e91d-426e-469e-b521-7ba71eb8f84e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:11.893428 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5e91d-426e-469e-b521-7ba71eb8f84e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:11.904942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 447/894] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:24:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:11.906884 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5e91d-426e-469e-b521-7ba71eb8f84e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:11.926843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984d348-747c-4639-8291-9ddd1db9f72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:11.927590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984d348-747c-4639-8291-9ddd1db9f72d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:11.927861 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984d348-747c-4639-8291-9ddd1db9f72d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:11.928083 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984d348-747c-4639-8291-9ddd1db9f72d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:11.940227 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984d348-747c-4639-8291-9ddd1db9f72d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-0984d348-747c-4639-8291-9ddd1db9f72d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:11.946803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:11.947739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:11.949051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:11.949051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:11.949051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:11.949051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:11.949051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:11.949051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:11.949051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:11.949051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:11.949051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:11.949051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:11.949051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:24:11.949051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:11.951042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 448/895] 10.222.0.62 () {56 vars in 1120 bytes} [Thu Sep 1 16:24:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:11.957417 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984d348-747c-4639-8291-9ddd1db9f72d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:12.015591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:12.040427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5e91d-426e-469e-b521-7ba71eb8f84e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:12.063445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984d348-747c-4639-8291-9ddd1db9f72d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:12.079454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:12.088254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e4053-c097-4cb1-90e4-bdc3d336cb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:12.088879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e4053-c097-4cb1-90e4-bdc3d336cb17 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:12.089150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e4053-c097-4cb1-90e4-bdc3d336cb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:12.089420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e4053-c097-4cb1-90e4-bdc3d336cb17 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:12.090033 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e4053-c097-4cb1-90e4-bdc3d336cb17 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:12.090357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e4053-c097-4cb1-90e4-bdc3d336cb17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:12.091706 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e4053-c097-4cb1-90e4-bdc3d336cb17 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:12.101945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 448/896] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:24:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:12.102931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e4053-c097-4cb1-90e4-bdc3d336cb17 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:12.108579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbc72c-8c8c-4e1d-95e6-212041278a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:12.109459 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbc72c-8c8c-4e1d-95e6-212041278a68 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:12.109956 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbc72c-8c8c-4e1d-95e6-212041278a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:12.110477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbc72c-8c8c-4e1d-95e6-212041278a68 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:12.111337 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbc72c-8c8c-4e1d-95e6-212041278a68 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:12.111864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbc72c-8c8c-4e1d-95e6-212041278a68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:12.113968 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbc72c-8c8c-4e1d-95e6-212041278a68 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:12.126465 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 449/897] 10.222.0.62 () {56 vars in 1115 bytes} [Thu Sep 1 16:24:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:12.132421 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbc72c-8c8c-4e1d-95e6-212041278a68 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:12.211084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e4053-c097-4cb1-90e4-bdc3d336cb17 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:12.255741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbc72c-8c8c-4e1d-95e6-212041278a68 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:12.258065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e019cd-0533-468b-959b-bbcaa8f3d124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:12.259527 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e019cd-0533-468b-959b-bbcaa8f3d124 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:12.260066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e019cd-0533-468b-959b-bbcaa8f3d124 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:12.260646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e019cd-0533-468b-959b-bbcaa8f3d124 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:12.261600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e019cd-0533-468b-959b-bbcaa8f3d124 None admin] RBAC: Adding query filter params (interface=public, service_id=d3acae88c1104c538cde837ef4a12e59, region_id=RegionOne) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:12.262115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e019cd-0533-468b-959b-bbcaa8f3d124 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:12.263787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e019cd-0533-468b-959b-bbcaa8f3d124 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:12.270342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 449/898] 10.222.0.62 () {56 vars in 1251 bytes} [Thu Sep 1 16:24:12 2022] GET /identity/v3/endpoints?service_id=d3acae88c1104c538cde837ef4a12e59&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:12.280009 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e019cd-0533-468b-959b-bbcaa8f3d124 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:12.294992 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6baff3-2087-4e33-837a-d1ed2c4dcdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:12.295816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:12.296494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6baff3-2087-4e33-837a-d1ed2c4dcdf6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:12.296945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6baff3-2087-4e33-837a-d1ed2c4dcdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:12.297381 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6baff3-2087-4e33-837a-d1ed2c4dcdf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:12.309723 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6baff3-2087-4e33-837a-d1ed2c4dcdf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:12.311409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6baff3-2087-4e33-837a-d1ed2c4dcdf6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-5f6baff3-2087-4e33-837a-d1ed2c4dcdf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:12.327239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:12.328313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:12.329221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:12.329221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:12.329221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:12.330965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 450/899] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:12.336930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6baff3-2087-4e33-837a-d1ed2c4dcdf6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:12.366717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:12.411185 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e019cd-0533-468b-959b-bbcaa8f3d124 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:12.448257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6baff3-2087-4e33-837a-d1ed2c4dcdf6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:12.452359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df396b7-b687-471c-8f66-7db5e73c44ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:12.452984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df396b7-b687-471c-8f66-7db5e73c44ad None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:12.453260 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df396b7-b687-471c-8f66-7db5e73c44ad None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:12.453524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df396b7-b687-471c-8f66-7db5e73c44ad None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:12.454126 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df396b7-b687-471c-8f66-7db5e73c44ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:12.455512 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df396b7-b687-471c-8f66-7db5e73c44ad None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:12.471924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 450/900] 10.222.0.62 () {56 vars in 1087 bytes} [Thu Sep 1 16:24:12 2022] GET /identity/v3/services => generated 1268 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:24:12.501126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e57d52-c24d-4556-a442-5031f3b0d602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:12.501754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e57d52-c24d-4556-a442-5031f3b0d602 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:12.502028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e57d52-c24d-4556-a442-5031f3b0d602 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:12.502295 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e57d52-c24d-4556-a442-5031f3b0d602 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:12.502948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e57d52-c24d-4556-a442-5031f3b0d602 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:12.503248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e57d52-c24d-4556-a442-5031f3b0d602 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:12.504677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e57d52-c24d-4556-a442-5031f3b0d602 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:12.519579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 451/901] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:12.526104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df396b7-b687-471c-8f66-7db5e73c44ad None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:12.617600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:12.646861 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df396b7-b687-471c-8f66-7db5e73c44ad None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:12.683879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d6bbe-f408-43d3-932b-9bf88a723ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:12.684554 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d6bbe-f408-43d3-932b-9bf88a723ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:12.684871 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d6bbe-f408-43d3-932b-9bf88a723ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:12.685141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d6bbe-f408-43d3-932b-9bf88a723ee9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:12.693401 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d6bbe-f408-43d3-932b-9bf88a723ee9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:12.694920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d6bbe-f408-43d3-932b-9bf88a723ee9 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-651d6bbe-f408-43d3-932b-9bf88a723ee9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:12.707511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:12.708873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:12.709792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:12.709792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:12.709792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:12.709792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:12.709792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:12.709792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:12.709792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:12.709792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:12.709792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:24:12.709792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:12.709792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 451/902] 10.222.0.62 () {56 vars in 1117 bytes} [Thu Sep 1 16:24:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:12.716894 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e57d52-c24d-4556-a442-5031f3b0d602 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:12.809980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:12.826419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e57d52-c24d-4556-a442-5031f3b0d602 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:12.861993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc0e82-3c7c-4fbb-9cc2-f198d415772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:12.862882 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc0e82-3c7c-4fbb-9cc2-f198d415772d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:12.863302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc0e82-3c7c-4fbb-9cc2-f198d415772d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:12.863716 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc0e82-3c7c-4fbb-9cc2-f198d415772d None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:12.864601 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc0e82-3c7c-4fbb-9cc2-f198d415772d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:12.865055 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc0e82-3c7c-4fbb-9cc2-f198d415772d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:12.866607 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc0e82-3c7c-4fbb-9cc2-f198d415772d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:12.885274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 452/903] 10.222.0.62 () {56 vars in 1149 bytes} [Thu Sep 1 16:24:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:12.891602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d6bbe-f408-43d3-932b-9bf88a723ee9 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:12.997505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d6bbe-f408-43d3-932b-9bf88a723ee9 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:13.031468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59617e38-6291-445c-b60c-62a5bfef74ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:13.032115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59617e38-6291-445c-b60c-62a5bfef74ce None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:13.032391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59617e38-6291-445c-b60c-62a5bfef74ce None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:13.032655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59617e38-6291-445c-b60c-62a5bfef74ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:13.036167 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:13.100529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59617e38-6291-445c-b60c-62a5bfef74ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:13.101990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59617e38-6291-445c-b60c-62a5bfef74ce None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:13.150110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e937a0-dae2-4c14-a9f0-8b8261ab5655 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:13.150308 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e937a0-dae2-4c14-a9f0-8b8261ab5655 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:13.150617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e937a0-dae2-4c14-a9f0-8b8261ab5655 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:13.155952 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 453/904] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:13.157529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada147fc-1750-4e92-a2a4-20b694500f44 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:13.157812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada147fc-1750-4e92-a2a4-20b694500f44 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:13.158087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada147fc-1750-4e92-a2a4-20b694500f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:13.176796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ada147fc-1750-4e92-a2a4-20b694500f44 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:13.178000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ada147fc-1750-4e92-a2a4-20b694500f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:13.210016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 452/905] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:12 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:24:13.215602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59617e38-6291-445c-b60c-62a5bfef74ce None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:13.254235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 454/906] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:13.255837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:13.319248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59617e38-6291-445c-b60c-62a5bfef74ce None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:13.353634 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3962d1b-dcf5-4aaf-978a-e9acb31b69ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:13.354903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3962d1b-dcf5-4aaf-978a-e9acb31b69ee None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:13.355181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3962d1b-dcf5-4aaf-978a-e9acb31b69ee None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:13.355446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3962d1b-dcf5-4aaf-978a-e9acb31b69ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:13.365747 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3962d1b-dcf5-4aaf-978a-e9acb31b69ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:13.367204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3962d1b-dcf5-4aaf-978a-e9acb31b69ee None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-a3962d1b-dcf5-4aaf-978a-e9acb31b69ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:13.381800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:13.382791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:13.383658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:13.383658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:13.383658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:13.383658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 453/907] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:13.393036 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada147fc-1750-4e92-a2a4-20b694500f44 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:13.474424 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:13.493010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada147fc-1750-4e92-a2a4-20b694500f44 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:13.518530 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3962d1b-dcf5-4aaf-978a-e9acb31b69ee None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:13.529330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62de6cf-c59b-4bc2-9327-18f45cbce3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:13.529950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62de6cf-c59b-4bc2-9327-18f45cbce3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:13.530222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62de6cf-c59b-4bc2-9327-18f45cbce3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:13.530821 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62de6cf-c59b-4bc2-9327-18f45cbce3b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:13.531474 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62de6cf-c59b-4bc2-9327-18f45cbce3b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:13.531772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62de6cf-c59b-4bc2-9327-18f45cbce3b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:13.533370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62de6cf-c59b-4bc2-9327-18f45cbce3b3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:13.547712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 455/908] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:13.554608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62de6cf-c59b-4bc2-9327-18f45cbce3b3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:13.621488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:13.640166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3962d1b-dcf5-4aaf-978a-e9acb31b69ee None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:13.662224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62de6cf-c59b-4bc2-9327-18f45cbce3b3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:13.680192 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d96c3b-7fdc-4c86-b168-b90f6ec536c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:13.681317 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d96c3b-7fdc-4c86-b168-b90f6ec536c7 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:13.681796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d96c3b-7fdc-4c86-b168-b90f6ec536c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:13.682275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d96c3b-7fdc-4c86-b168-b90f6ec536c7 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:13.683137 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d96c3b-7fdc-4c86-b168-b90f6ec536c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:13.684859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d96c3b-7fdc-4c86-b168-b90f6ec536c7 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:13.698876 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7f8c8-1f19-4300-8603-e95f78104ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:13.699516 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7f8c8-1f19-4300-8603-e95f78104ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:13.699788 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7f8c8-1f19-4300-8603-e95f78104ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:13.700082 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7f8c8-1f19-4300-8603-e95f78104ad0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:13.702149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:13.712134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 454/909] 10.222.0.62 () {60 vars in 1141 bytes} [Thu Sep 1 16:24:13 2022] POST /identity/v3/services => generated 241 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:24:13.714914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7f8c8-1f19-4300-8603-e95f78104ad0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:13.715085 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7f8c8-1f19-4300-8603-e95f78104ad0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e2d7f8c8-1f19-4300-8603-e95f78104ad0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:13.729019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:13.729972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:13.731529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:13.731529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:13.731529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:13.731529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:13.731529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:13.731529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:13.731529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:13.731529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:13.731529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 456/910] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:13.736328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d96c3b-7fdc-4c86-b168-b90f6ec536c7 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:13.766938 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df391e0-177b-4858-8106-e840175fdaee None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:13.767264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df391e0-177b-4858-8106-e840175fdaee None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:13.767469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df391e0-177b-4858-8106-e840175fdaee None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:13.769514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 457/911] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:13.776112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20723c-14be-4092-8a90-3a12d622c616 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:13.776754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20723c-14be-4092-8a90-3a12d622c616 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:13.777223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20723c-14be-4092-8a90-3a12d622c616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:13.797024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-aa20723c-14be-4092-8a90-3a12d622c616 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:13.798607 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-aa20723c-14be-4092-8a90-3a12d622c616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:13.870217 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d96c3b-7fdc-4c86-b168-b90f6ec536c7 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:13.893528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 458/912] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:13.919271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1df96-4139-4f57-af7b-683555792e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:13.919944 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1df96-4139-4f57-af7b-683555792e0d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:13.920196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1df96-4139-4f57-af7b-683555792e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:13.920460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1df96-4139-4f57-af7b-683555792e0d None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:13.921215 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1df96-4139-4f57-af7b-683555792e0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:13.921519 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1df96-4139-4f57-af7b-683555792e0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:13.922985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1df96-4139-4f57-af7b-683555792e0d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:13.936503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 455/913] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:13.942561 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20723c-14be-4092-8a90-3a12d622c616 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:14.046478 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20723c-14be-4092-8a90-3a12d622c616 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:14.070796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:14.083228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336c316-c1f4-4012-96c6-e37067afce37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:14.083844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336c316-c1f4-4012-96c6-e37067afce37 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:14.084135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336c316-c1f4-4012-96c6-e37067afce37 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:14.084400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336c316-c1f4-4012-96c6-e37067afce37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:14.085121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336c316-c1f4-4012-96c6-e37067afce37 None admin] RBAC: Adding query filter params (role.id=814881b0c50648f4a0edcdf280dd29c7, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=24705faaf5054802a556749c7ba82ee8) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:14.085429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336c316-c1f4-4012-96c6-e37067afce37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:14.086855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336c316-c1f4-4012-96c6-e37067afce37 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:14.093031 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 459/914] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:13 2022] GET /identity/v3/role_assignments?user.id=24705faaf5054802a556749c7ba82ee8&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=814881b0c50648f4a0edcdf280dd29c7 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:14.157482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1df96-4139-4f57-af7b-683555792e0d None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:14.160615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:14.272440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1df96-4139-4f57-af7b-683555792e0d None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:14.308457 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556eea2a-3b55-4e09-9842-facd445418c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:14.309342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eea2a-3b55-4e09-9842-facd445418c3 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:14.309764 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eea2a-3b55-4e09-9842-facd445418c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:14.310175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eea2a-3b55-4e09-9842-facd445418c3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:14.310983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556eea2a-3b55-4e09-9842-facd445418c3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:14.312534 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556eea2a-3b55-4e09-9842-facd445418c3 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-556eea2a-3b55-4e09-9842-facd445418c3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:14.322446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:14.324047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:14.325144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:14.325144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:14.325144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:14.325144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:14.325144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:14.325144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:14.325144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:14.325144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:14.325144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:14.325144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:14.325144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:14.325144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:14.325144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:24:14.325144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:14.327623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 456/915] 10.222.0.62 () {56 vars in 1132 bytes} [Thu Sep 1 16:24:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:14.333744 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336c316-c1f4-4012-96c6-e37067afce37 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:14.435575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336c316-c1f4-4012-96c6-e37067afce37 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:14.471780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b1c08-6a82-43ae-8c5e-8552b765e0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:14.472425 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b1c08-6a82-43ae-8c5e-8552b765e0ae None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:14.472701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b1c08-6a82-43ae-8c5e-8552b765e0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:14.472969 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b1c08-6a82-43ae-8c5e-8552b765e0ae None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:14.473606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b1c08-6a82-43ae-8c5e-8552b765e0ae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:14.473906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b1c08-6a82-43ae-8c5e-8552b765e0ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:14.475284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b1c08-6a82-43ae-8c5e-8552b765e0ae None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:14.480799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 460/916] 10.222.0.62 () {56 vars in 1126 bytes} [Thu Sep 1 16:24:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:14.486711 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556eea2a-3b55-4e09-9842-facd445418c3 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:14.503558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df47fe-7efd-414a-9210-66745b45b48d None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:14.503866 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df47fe-7efd-414a-9210-66745b45b48d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:14.504149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df47fe-7efd-414a-9210-66745b45b48d None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:14.505754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 461/917] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:14.510717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c75f7-a711-484b-8eea-a7950eef6fef None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:14.511003 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c75f7-a711-484b-8eea-a7950eef6fef None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:14.511278 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c75f7-a711-484b-8eea-a7950eef6fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:14.533600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3d3c75f7-a711-484b-8eea-a7950eef6fef None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:14.535447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3d3c75f7-a711-484b-8eea-a7950eef6fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:14.561579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:14.587951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:14.603722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556eea2a-3b55-4e09-9842-facd445418c3 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:14.610138 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 462/918] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:14.644053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47585e27-f0cf-4770-9b5a-3ed40b076ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:14.644998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47585e27-f0cf-4770-9b5a-3ed40b076ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:14.645416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47585e27-f0cf-4770-9b5a-3ed40b076ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:14.645824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47585e27-f0cf-4770-9b5a-3ed40b076ac1 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:14.646588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47585e27-f0cf-4770-9b5a-3ed40b076ac1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:14.647142 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47585e27-f0cf-4770-9b5a-3ed40b076ac1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:14.648746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47585e27-f0cf-4770-9b5a-3ed40b076ac1 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:14.654451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 457/919] 10.222.0.62 () {56 vars in 1126 bytes} [Thu Sep 1 16:24:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:14.660318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c75f7-a711-484b-8eea-a7950eef6fef None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:14.779778 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c75f7-a711-484b-8eea-a7950eef6fef None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:14.815625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55bc6e-909f-42ae-ade5-50650cf564b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:14.816289 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55bc6e-909f-42ae-ade5-50650cf564b7 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:14.816566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55bc6e-909f-42ae-ade5-50650cf564b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:14.816833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55bc6e-909f-42ae-ade5-50650cf564b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:14.817520 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55bc6e-909f-42ae-ade5-50650cf564b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:14.818908 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55bc6e-909f-42ae-ade5-50650cf564b7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:14.845221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 463/920] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:24:14 2022] POST /identity/v3/endpoints => generated 347 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:24:14.927565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47585e27-f0cf-4770-9b5a-3ed40b076ac1 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:15.036266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47585e27-f0cf-4770-9b5a-3ed40b076ac1 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:15.065985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:15.077843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd5b1e-77b5-4650-a5fd-e1b6eb30c4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:15.081384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd5b1e-77b5-4650-a5fd-e1b6eb30c4cd None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:15.081384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd5b1e-77b5-4650-a5fd-e1b6eb30c4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:15.081384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd5b1e-77b5-4650-a5fd-e1b6eb30c4cd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:15.081384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd5b1e-77b5-4650-a5fd-e1b6eb30c4cd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:15.081384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd5b1e-77b5-4650-a5fd-e1b6eb30c4cd None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-46cd5b1e-77b5-4650-a5fd-e1b6eb30c4cd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:15.091460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:15.092701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:15.093751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:15.093751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:15.093751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:15.093751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:15.093751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:15.093751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:15.093751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:15.093751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:15.093751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:15.093751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:15.093751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:15.093751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:15.093751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:24:15.093751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:15.096535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 458/921] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:24:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:15.102537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55bc6e-909f-42ae-ade5-50650cf564b7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:15.149800 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:15.232808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55bc6e-909f-42ae-ade5-50650cf564b7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:15.282504 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66364fc1-dd58-409d-9055-5bd63bbf0f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:15.283462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66364fc1-dd58-409d-9055-5bd63bbf0f6e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:15.284044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66364fc1-dd58-409d-9055-5bd63bbf0f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:15.284497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66364fc1-dd58-409d-9055-5bd63bbf0f6e None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:15.285344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66364fc1-dd58-409d-9055-5bd63bbf0f6e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:15.285961 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66364fc1-dd58-409d-9055-5bd63bbf0f6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:15.288636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66364fc1-dd58-409d-9055-5bd63bbf0f6e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:15.298034 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 464/922] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:24:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:15.304086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd5b1e-77b5-4650-a5fd-e1b6eb30c4cd None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:15.414692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd5b1e-77b5-4650-a5fd-e1b6eb30c4cd None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:15.450730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93529a6-43e3-4aa9-98a1-ed86bd9274e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:15.451655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93529a6-43e3-4aa9-98a1-ed86bd9274e5 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:15.452168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93529a6-43e3-4aa9-98a1-ed86bd9274e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:15.452628 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93529a6-43e3-4aa9-98a1-ed86bd9274e5 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:15.453493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93529a6-43e3-4aa9-98a1-ed86bd9274e5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:15.453967 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93529a6-43e3-4aa9-98a1-ed86bd9274e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:15.455561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93529a6-43e3-4aa9-98a1-ed86bd9274e5 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:15.461622 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 459/923] 10.222.0.62 () {56 vars in 1114 bytes} [Thu Sep 1 16:24:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:15.467836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66364fc1-dd58-409d-9055-5bd63bbf0f6e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:15.581108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66364fc1-dd58-409d-9055-5bd63bbf0f6e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:15.584032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:15.626948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee171d0-ea95-4a04-8acb-94b1a0325500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:15.627568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee171d0-ea95-4a04-8acb-94b1a0325500 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:15.627903 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee171d0-ea95-4a04-8acb-94b1a0325500 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:15.628191 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee171d0-ea95-4a04-8acb-94b1a0325500 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:15.628872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee171d0-ea95-4a04-8acb-94b1a0325500 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:15.630242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee171d0-ea95-4a04-8acb-94b1a0325500 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:15.652855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 465/924] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:24:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:24:15.768793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc6b86-0d07-4ab9-8493-b00d22160669 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:15.769106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc6b86-0d07-4ab9-8493-b00d22160669 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:15.769350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc6b86-0d07-4ab9-8493-b00d22160669 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:15.771069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 460/925] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:15.778108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9120b-2411-486f-a54f-e632b3a12547 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:15.778582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9120b-2411-486f-a54f-e632b3a12547 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:15.778954 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9120b-2411-486f-a54f-e632b3a12547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:15.799712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3fe9120b-2411-486f-a54f-e632b3a12547 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:15.801027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3fe9120b-2411-486f-a54f-e632b3a12547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:15.879949 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 466/926] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:16.151852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020130cd-22fd-4952-959c-361bdbcebc65 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.152187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020130cd-22fd-4952-959c-361bdbcebc65 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.152508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020130cd-22fd-4952-959c-361bdbcebc65 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.154187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 461/927] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:16.157533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9120b-2411-486f-a54f-e632b3a12547 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:16.159460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a274fb-83c7-4775-a200-21163778054e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.159738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a274fb-83c7-4775-a200-21163778054e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.160062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a274fb-83c7-4775-a200-21163778054e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.178533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-90a274fb-83c7-4775-a200-21163778054e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:16.179731 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-90a274fb-83c7-4775-a200-21163778054e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:16.252293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:16.264007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 462/928] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:16.285962 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9120b-2411-486f-a54f-e632b3a12547 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:16.289463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:16.325218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74539f43-4aa4-4eb4-8bf6-202489647eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:16.325869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74539f43-4aa4-4eb4-8bf6-202489647eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.326147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74539f43-4aa4-4eb4-8bf6-202489647eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.326420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74539f43-4aa4-4eb4-8bf6-202489647eb6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.327007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74539f43-4aa4-4eb4-8bf6-202489647eb6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:16.328420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74539f43-4aa4-4eb4-8bf6-202489647eb6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-74539f43-4aa4-4eb4-8bf6-202489647eb6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:16.338357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:16.339441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:16.340804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 467/929] 10.222.0.62 () {56 vars in 1126 bytes} [Thu Sep 1 16:24:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:16.345868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a274fb-83c7-4775-a200-21163778054e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:16.451079 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a274fb-83c7-4775-a200-21163778054e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:16.487568 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db484853-859f-40c8-a5c6-8f739fcd2774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:16.488211 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db484853-859f-40c8-a5c6-8f739fcd2774 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.488490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db484853-859f-40c8-a5c6-8f739fcd2774 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.488760 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db484853-859f-40c8-a5c6-8f739fcd2774 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.489399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db484853-859f-40c8-a5c6-8f739fcd2774 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:16.489755 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db484853-859f-40c8-a5c6-8f739fcd2774 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:16.491494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db484853-859f-40c8-a5c6-8f739fcd2774 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:16.497998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 463/930] 10.222.0.62 () {56 vars in 1122 bytes} [Thu Sep 1 16:24:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:16.503749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74539f43-4aa4-4eb4-8bf6-202489647eb6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:16.610949 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74539f43-4aa4-4eb4-8bf6-202489647eb6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:16.619016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db484853-859f-40c8-a5c6-8f739fcd2774 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:16.655134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65f505-72d0-4b40-8548-f0f5b9a255fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:16.656120 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65f505-72d0-4b40-8548-f0f5b9a255fc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.656602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65f505-72d0-4b40-8548-f0f5b9a255fc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.657062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65f505-72d0-4b40-8548-f0f5b9a255fc None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.657947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65f505-72d0-4b40-8548-f0f5b9a255fc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:16.658482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65f505-72d0-4b40-8548-f0f5b9a255fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:16.660117 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65f505-72d0-4b40-8548-f0f5b9a255fc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:16.665870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 468/931] 10.222.0.62 () {56 vars in 1122 bytes} [Thu Sep 1 16:24:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:16.671929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65f505-72d0-4b40-8548-f0f5b9a255fc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:16.682831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:16.693629 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:16.747978 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db484853-859f-40c8-a5c6-8f739fcd2774 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:16.797533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329f3df-274d-4efb-b979-ed4f54d0da58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:16.798525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329f3df-274d-4efb-b979-ed4f54d0da58 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.798944 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329f3df-274d-4efb-b979-ed4f54d0da58 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.799373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329f3df-274d-4efb-b979-ed4f54d0da58 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.799497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65f505-72d0-4b40-8548-f0f5b9a255fc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:16.813071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329f3df-274d-4efb-b979-ed4f54d0da58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-9329f3df-274d-4efb-b979-ed4f54d0da58 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:16.818716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:16.819857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:16.820864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:16.820864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:16.820864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:16.820864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:16.820864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:16.820864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:16.820864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:16.820864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:16.820864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:16.820864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:16.820864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:24:16.820864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:16.828510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 464/932] 10.222.0.62 () {56 vars in 1120 bytes} [Thu Sep 1 16:24:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:16.830155 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329f3df-274d-4efb-b979-ed4f54d0da58 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:16.838705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d25bad-1f90-47b2-a532-72a2c75d0298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:16.839324 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d25bad-1f90-47b2-a532-72a2c75d0298 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.839596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d25bad-1f90-47b2-a532-72a2c75d0298 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.839860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d25bad-1f90-47b2-a532-72a2c75d0298 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.840587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d25bad-1f90-47b2-a532-72a2c75d0298 None admin] RBAC: Adding query filter params (interface=public, service_id=cbf823fd81844e91b44cbf4b43a53719, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:16.840888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d25bad-1f90-47b2-a532-72a2c75d0298 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:16.842241 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d25bad-1f90-47b2-a532-72a2c75d0298 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:16.849138 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 469/933] 10.222.0.62 () {56 vars in 1251 bytes} [Thu Sep 1 16:24:16 2022] GET /identity/v3/endpoints?service_id=cbf823fd81844e91b44cbf4b43a53719&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:16.854637 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d25bad-1f90-47b2-a532-72a2c75d0298 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:16.940925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329f3df-274d-4efb-b979-ed4f54d0da58 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:16.942937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:16.950942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:16.970394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d25bad-1f90-47b2-a532-72a2c75d0298 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:16.977957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3492ba-3cdf-4fca-93c3-28dc6c34a854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:16.978624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3492ba-3cdf-4fca-93c3-28dc6c34a854 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.978894 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3492ba-3cdf-4fca-93c3-28dc6c34a854 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.979158 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3492ba-3cdf-4fca-93c3-28dc6c34a854 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.979821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3492ba-3cdf-4fca-93c3-28dc6c34a854 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:16.980155 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3492ba-3cdf-4fca-93c3-28dc6c34a854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:16.981547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3492ba-3cdf-4fca-93c3-28dc6c34a854 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:16.995106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 465/934] 10.222.0.62 () {56 vars in 1115 bytes} [Thu Sep 1 16:24:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:16.996446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfb1f8-8f58-4e14-9f51-b21987a1b00e None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:16.996734 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfb1f8-8f58-4e14-9f51-b21987a1b00e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:16.997020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfb1f8-8f58-4e14-9f51-b21987a1b00e None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:16.998714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 466/935] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:16.999335 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfb1f8-8f58-4e14-9f51-b21987a1b00e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:17.009854 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec69ff-b1c7-44bb-bf5b-af0c4d46cf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:17.010479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec69ff-b1c7-44bb-bf5b-af0c4d46cf0c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.010758 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec69ff-b1c7-44bb-bf5b-af0c4d46cf0c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.011032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec69ff-b1c7-44bb-bf5b-af0c4d46cf0c None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.011646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec69ff-b1c7-44bb-bf5b-af0c4d46cf0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:17.013124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec69ff-b1c7-44bb-bf5b-af0c4d46cf0c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:17.027919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 470/936] 10.222.0.62 () {56 vars in 1087 bytes} [Thu Sep 1 16:24:16 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:24:17.027919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6e3b7-5a41-4d0c-bd67-8a718e1d2577 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.027919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6e3b7-5a41-4d0c-bd67-8a718e1d2577 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.027919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6e3b7-5a41-4d0c-bd67-8a718e1d2577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.041052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-efa6e3b7-5a41-4d0c-bd67-8a718e1d2577 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:17.042363 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-efa6e3b7-5a41-4d0c-bd67-8a718e1d2577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:17.128347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 471/937] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:17.141319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfb1f8-8f58-4e14-9f51-b21987a1b00e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:17.178649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13ac49-70b7-44a6-ae71-e1daf02efa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:17.179313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13ac49-70b7-44a6-ae71-e1daf02efa03 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.179589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13ac49-70b7-44a6-ae71-e1daf02efa03 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.179855 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13ac49-70b7-44a6-ae71-e1daf02efa03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.190893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13ac49-70b7-44a6-ae71-e1daf02efa03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:17.192405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13ac49-70b7-44a6-ae71-e1daf02efa03 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-fb13ac49-70b7-44a6-ae71-e1daf02efa03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:17.206983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:17.208375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:17.209569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:17.209569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:17.209569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:17.209569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 467/938] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:17.216046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6e3b7-5a41-4d0c-bd67-8a718e1d2577 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:17.291728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:17.324093 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6e3b7-5a41-4d0c-bd67-8a718e1d2577 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:17.360524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af76b3c-00d8-4688-9bb4-125ed16f557c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:17.361133 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af76b3c-00d8-4688-9bb4-125ed16f557c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.361409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af76b3c-00d8-4688-9bb4-125ed16f557c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.361677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af76b3c-00d8-4688-9bb4-125ed16f557c None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.362318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af76b3c-00d8-4688-9bb4-125ed16f557c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:17.362614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af76b3c-00d8-4688-9bb4-125ed16f557c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:17.364076 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af76b3c-00d8-4688-9bb4-125ed16f557c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:17.381132 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 472/939] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:17.382679 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13ac49-70b7-44a6-ae71-e1daf02efa03 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:17.397774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af76b3c-00d8-4688-9bb4-125ed16f557c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:17.413461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:17.505755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af76b3c-00d8-4688-9bb4-125ed16f557c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:17.511368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13ac49-70b7-44a6-ae71-e1daf02efa03 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:17.540258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:17.550732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a815b8c-edde-48b3-baff-8637b81eb3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:17.551744 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a815b8c-edde-48b3-baff-8637b81eb3cf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.552497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a815b8c-edde-48b3-baff-8637b81eb3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.553238 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a815b8c-edde-48b3-baff-8637b81eb3cf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.554152 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a815b8c-edde-48b3-baff-8637b81eb3cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:17.556393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a815b8c-edde-48b3-baff-8637b81eb3cf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:17.557242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd1cd2-7b37-4e82-9f86-7887e7912b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:17.557888 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd1cd2-7b37-4e82-9f86-7887e7912b75 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.558169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd1cd2-7b37-4e82-9f86-7887e7912b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.558440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd1cd2-7b37-4e82-9f86-7887e7912b75 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.566586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd1cd2-7b37-4e82-9f86-7887e7912b75 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:17.568131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd1cd2-7b37-4e82-9f86-7887e7912b75 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9a815b8c-edde-48b3-baff-8637b81eb3cf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:17.568985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:17.570183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:17.571235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:17.571235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:17.571235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:17.571235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:17.571235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:17.571235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:17.571235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:17.571235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:17.571235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:17.571235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:17.571235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:17.571235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:17.571235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:24:17.571235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:17.573651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 473/940] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:17.580787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a815b8c-edde-48b3-baff-8637b81eb3cf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-b5dd1cd2-7b37-4e82-9f86-7887e7912b75 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:17.588940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:17.589940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:17.590883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:17.590883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:17.590883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:17.590883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:17.590883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:17.590883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:17.590883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:17.590883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:17.590883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:24:17.590883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:17.595506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 468/941] 10.222.0.62 () {56 vars in 1117 bytes} [Thu Sep 1 16:24:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:17.596857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd1cd2-7b37-4e82-9f86-7887e7912b75 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:17.703741 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:17.725681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd1cd2-7b37-4e82-9f86-7887e7912b75 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:17.729329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a815b8c-edde-48b3-baff-8637b81eb3cf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:17.768380 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b79746-e2a0-4a9a-b798-01e8aad1b68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:17.768995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b79746-e2a0-4a9a-b798-01e8aad1b68d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.769272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b79746-e2a0-4a9a-b798-01e8aad1b68d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.769539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b79746-e2a0-4a9a-b798-01e8aad1b68d None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.770275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b79746-e2a0-4a9a-b798-01e8aad1b68d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:17.770570 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b79746-e2a0-4a9a-b798-01e8aad1b68d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:17.771178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82965565-0e4e-46e8-a17f-e31f9aa93308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:17.771787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82965565-0e4e-46e8-a17f-e31f9aa93308 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.772029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b79746-e2a0-4a9a-b798-01e8aad1b68d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:17.772135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82965565-0e4e-46e8-a17f-e31f9aa93308 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.772367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82965565-0e4e-46e8-a17f-e31f9aa93308 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.772993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82965565-0e4e-46e8-a17f-e31f9aa93308 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:17.773295 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82965565-0e4e-46e8-a17f-e31f9aa93308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:17.775650 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82965565-0e4e-46e8-a17f-e31f9aa93308 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:17.783986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 474/942] 10.222.0.62 () {56 vars in 1112 bytes} [Thu Sep 1 16:24:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:17.783986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1541f-23eb-4e43-a454-0447ac0ff4f9 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.783986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1541f-23eb-4e43-a454-0447ac0ff4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.783986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1541f-23eb-4e43-a454-0447ac0ff4f9 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.783986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 475/943] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:17.786214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1541f-23eb-4e43-a454-0447ac0ff4f9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:17.796271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 469/944] 10.222.0.62 () {56 vars in 1149 bytes} [Thu Sep 1 16:24:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:17.797730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738c246-47f3-40fe-81b8-45bc47369b87 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.798078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738c246-47f3-40fe-81b8-45bc47369b87 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.798778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738c246-47f3-40fe-81b8-45bc47369b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.817119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-5738c246-47f3-40fe-81b8-45bc47369b87 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:17.818342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5738c246-47f3-40fe-81b8-45bc47369b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:17.852408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:17.895929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 470/945] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:17.895929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738c246-47f3-40fe-81b8-45bc47369b87 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:17.914792 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1541f-23eb-4e43-a454-0447ac0ff4f9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:17.963490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cded5-6f34-42d9-833e-54733677c369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:17.964361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cded5-6f34-42d9-833e-54733677c369 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:17.964648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cded5-6f34-42d9-833e-54733677c369 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:17.964917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cded5-6f34-42d9-833e-54733677c369 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:17.965549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38cded5-6f34-42d9-833e-54733677c369 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:17.966412 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38cded5-6f34-42d9-833e-54733677c369 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:17.968169 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38cded5-6f34-42d9-833e-54733677c369 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:17.969485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:17.974942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 476/946] 10.222.0.62 () {56 vars in 1112 bytes} [Thu Sep 1 16:24:17 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:18.012733 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738c246-47f3-40fe-81b8-45bc47369b87 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:18.051579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85912e1-ce99-4bd3-8a77-329bd754589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:18.052276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85912e1-ce99-4bd3-8a77-329bd754589d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:18.052548 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85912e1-ce99-4bd3-8a77-329bd754589d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:18.052813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85912e1-ce99-4bd3-8a77-329bd754589d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:18.064370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85912e1-ce99-4bd3-8a77-329bd754589d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:18.065883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85912e1-ce99-4bd3-8a77-329bd754589d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-d85912e1-ce99-4bd3-8a77-329bd754589d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:18.082303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:18.083700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:18.085222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:18.085222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:18.085222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:18.085222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 471/947] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:18.092156 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cded5-6f34-42d9-833e-54733677c369 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:18.162557 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85912e1-ce99-4bd3-8a77-329bd754589d None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:18.217336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cded5-6f34-42d9-833e-54733677c369 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:18.248740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:18.267421 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac0394-59d6-42ba-826b-fc8574a7269f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:18.268065 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac0394-59d6-42ba-826b-fc8574a7269f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:18.268340 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac0394-59d6-42ba-826b-fc8574a7269f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:18.268606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac0394-59d6-42ba-826b-fc8574a7269f None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:18.269501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac0394-59d6-42ba-826b-fc8574a7269f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:18.269800 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac0394-59d6-42ba-826b-fc8574a7269f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:18.271540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac0394-59d6-42ba-826b-fc8574a7269f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:18.286721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85912e1-ce99-4bd3-8a77-329bd754589d None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:18.287919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 477/948] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:18.293576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac0394-59d6-42ba-826b-fc8574a7269f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:18.321112 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:18.336157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae30d20-b887-48fd-a358-fe6848a106f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:18.336778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae30d20-b887-48fd-a358-fe6848a106f6 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:18.337175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae30d20-b887-48fd-a358-fe6848a106f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:18.337501 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae30d20-b887-48fd-a358-fe6848a106f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:18.338238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae30d20-b887-48fd-a358-fe6848a106f6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:18.339771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae30d20-b887-48fd-a358-fe6848a106f6 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:18.343519 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: WARNING keystone.server.flask.common [None req-fae30d20-b887-48fd-a358-fe6848a106f6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 16:24:18.368235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.sql.core [None req-fae30d20-b887-48fd-a358-fe6848a106f6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 16:24:18.368235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 16:24:18.368235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [parameters: {'id': 'a7cecd2539844046a4dcbad238433b49', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 16:24:18.368235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81523) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 16:24:18.378071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-fae30d20-b887-48fd-a358-fe6848a106f6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a7cecd2539844046a4dcbad238433b49.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a7cecd2539844046a4dcbad238433b49. Sep 01 16:24:18.378071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:18.378071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:24:18.378071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:24:18.378071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:24:18.378071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:24:18.378071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:24:18.378071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:24:18.378071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:24:18.378071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:24:18.378071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:24:18.378071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:24:18.378071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application result.read() Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 16:24:18.378988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application yield resource Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise self.value Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application session.commit() Sep 01 16:24:18.380398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.session.flush() Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application compat.raise_( Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 16:24:18.381505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 16:24:18.382394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application result.read() Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:24:18.383990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [parameters: {'id': 'a7cecd2539844046a4dcbad238433b49', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:18.386219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:18.388101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:18.390269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:18.390269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 16:24:18.390269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 16:24:18.390269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:18.390269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:18.390269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 16:24:18.390269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 16:24:18.390269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a7cecd2539844046a4dcbad238433b49. Sep 01 16:24:18.390269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:18.390269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 472/949] 10.222.0.62 () {60 vars in 1140 bytes} [Thu Sep 1 16:24:18 2022] POST /identity/v3/projects => generated 262 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:18.390269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae30d20-b887-48fd-a358-fe6848a106f6 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:18.418529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac0394-59d6-42ba-826b-fc8574a7269f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:18.459178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e51c90-d140-47dc-97bb-96c8e2a6c5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:18.460273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e51c90-d140-47dc-97bb-96c8e2a6c5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:18.460895 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e51c90-d140-47dc-97bb-96c8e2a6c5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:18.461424 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e51c90-d140-47dc-97bb-96c8e2a6c5c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:18.471725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e51c90-d140-47dc-97bb-96c8e2a6c5c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:18.474422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e51c90-d140-47dc-97bb-96c8e2a6c5c5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-f5e51c90-d140-47dc-97bb-96c8e2a6c5c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:18.490454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:18.491401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:18.492333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:18.492333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:18.492333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:18.492333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:18.492333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:18.492333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:18.492333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:18.492333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:18.495248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 478/950] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:18.500574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e51c90-d140-47dc-97bb-96c8e2a6c5c5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:18.532289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae30d20-b887-48fd-a358-fe6848a106f6 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:18.558869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:18.581656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e30c2-6826-43dd-9b44-a69fb6543941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:18.583645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e30c2-6826-43dd-9b44-a69fb6543941 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:18.584162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e30c2-6826-43dd-9b44-a69fb6543941 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:18.584629 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e30c2-6826-43dd-9b44-a69fb6543941 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:18.594821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e30c2-6826-43dd-9b44-a69fb6543941 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:18.596900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e30c2-6826-43dd-9b44-a69fb6543941 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-7d0e30c2-6826-43dd-9b44-a69fb6543941 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:18.614315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:18.615226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:18.616154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:18.616154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:18.616154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:18.616154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:18.616154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:18.616154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:18.616154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:18.616154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:18.617456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 473/951] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:18.625237 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e30c2-6826-43dd-9b44-a69fb6543941 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:18.629660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e51c90-d140-47dc-97bb-96c8e2a6c5c5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:18.664345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc475e-e3eb-46fc-8bfd-4f832e47d0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:18.664956 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc475e-e3eb-46fc-8bfd-4f832e47d0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:18.665344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc475e-e3eb-46fc-8bfd-4f832e47d0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:18.665694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc475e-e3eb-46fc-8bfd-4f832e47d0e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:18.666413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc475e-e3eb-46fc-8bfd-4f832e47d0e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:18.666709 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc475e-e3eb-46fc-8bfd-4f832e47d0e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:18.668150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc475e-e3eb-46fc-8bfd-4f832e47d0e2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:18.678511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:18.685555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 479/952] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:18.692974 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc475e-e3eb-46fc-8bfd-4f832e47d0e2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:18.759729 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e30c2-6826-43dd-9b44-a69fb6543941 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:18.797875 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ec4e5-e537-459a-9a20-060cb650f813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:18.798489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ec4e5-e537-459a-9a20-060cb650f813 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:18.798766 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ec4e5-e537-459a-9a20-060cb650f813 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:18.799075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ec4e5-e537-459a-9a20-060cb650f813 None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:18.799580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc475e-e3eb-46fc-8bfd-4f832e47d0e2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:18.799811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ec4e5-e537-459a-9a20-060cb650f813 None admin] RBAC: Adding query filter params (name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:18.800164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ec4e5-e537-459a-9a20-060cb650f813 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:18.801195 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:18.802797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ec4e5-e537-459a-9a20-060cb650f813 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:18.827087 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 474/953] 10.222.0.62 () {56 vars in 1112 bytes} [Thu Sep 1 16:24:18 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:18.842003 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e6334-df0e-47a6-bec0-54d4c4bab8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:18.842698 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e6334-df0e-47a6-bec0-54d4c4bab8ee None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:18.842969 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e6334-df0e-47a6-bec0-54d4c4bab8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:18.843231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e6334-df0e-47a6-bec0-54d4c4bab8ee None admin] PATH_INFO: `/v3/projects/425048ffd13448cd8fe815333a9b798c/users/24705faaf5054802a556749c7ba82ee8/roles/814881b0c50648f4a0edcdf280dd29c7` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:18.845567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e6334-df0e-47a6-bec0-54d4c4bab8ee None admin] RBAC: Authorizing `identity:create_grant(project_id=425048ffd13448cd8fe815333a9b798c, user_id=24705faaf5054802a556749c7ba82ee8, role_id=814881b0c50648f4a0edcdf280dd29c7)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:18.846966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e6334-df0e-47a6-bec0-54d4c4bab8ee None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:18.864567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 480/954] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:24:18 2022] PUT /identity/v3/projects/425048ffd13448cd8fe815333a9b798c/users/24705faaf5054802a556749c7ba82ee8/roles/814881b0c50648f4a0edcdf280dd29c7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:24:19.060532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6572245-24eb-4cd8-bb2d-ac47cf635c86 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:19.060981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6572245-24eb-4cd8-bb2d-ac47cf635c86 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:19.061343 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6572245-24eb-4cd8-bb2d-ac47cf635c86 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:19.063040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 475/955] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:19.068653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4276730-c0eb-43c4-9e73-179eb899fcae None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:19.068939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4276730-c0eb-43c4-9e73-179eb899fcae None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:19.069211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4276730-c0eb-43c4-9e73-179eb899fcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:19.089774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e4276730-c0eb-43c4-9e73-179eb899fcae None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:19.090960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e4276730-c0eb-43c4-9e73-179eb899fcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:19.163008 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 481/956] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:19.438142 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6572245-24eb-4cd8-bb2d-ac47cf635c86 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:19.540410 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:19.546493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6572245-24eb-4cd8-bb2d-ac47cf635c86 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:19.584226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d3368-9075-45d4-b255-d13ec6bb5846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:19.585355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d3368-9075-45d4-b255-d13ec6bb5846 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:19.585873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d3368-9075-45d4-b255-d13ec6bb5846 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:19.586385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d3368-9075-45d4-b255-d13ec6bb5846 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:19.587218 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d3368-9075-45d4-b255-d13ec6bb5846 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:19.588827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d3368-9075-45d4-b255-d13ec6bb5846 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-2e3d3368-9075-45d4-b255-d13ec6bb5846 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:19.598392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:19.599489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:19.600513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:19.600513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:19.600513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:19.600513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:19.600513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:19.600513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:19.600513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:19.600513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:19.600513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:19.600513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:19.600513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:19.600513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:19.600513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:19.600513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:19.603059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 476/957] 10.222.0.62 () {56 vars in 1126 bytes} [Thu Sep 1 16:24:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:19.608590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4276730-c0eb-43c4-9e73-179eb899fcae None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:19.703355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:19.712746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4276730-c0eb-43c4-9e73-179eb899fcae None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:19.747178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d89d84-9313-4820-9521-15885d8cfcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:19.747782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d89d84-9313-4820-9521-15885d8cfcb5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:19.748083 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d89d84-9313-4820-9521-15885d8cfcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:19.748347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d89d84-9313-4820-9521-15885d8cfcb5 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:19.748966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d89d84-9313-4820-9521-15885d8cfcb5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:19.749259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d89d84-9313-4820-9521-15885d8cfcb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:19.750620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d89d84-9313-4820-9521-15885d8cfcb5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:19.755775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 482/958] 10.222.0.62 () {56 vars in 1122 bytes} [Thu Sep 1 16:24:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:19.761810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d3368-9075-45d4-b255-d13ec6bb5846 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:19.862794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d3368-9075-45d4-b255-d13ec6bb5846 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:19.901609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076df44-561d-4772-81be-db09848f73f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:19.902246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076df44-561d-4772-81be-db09848f73f0 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:19.902523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076df44-561d-4772-81be-db09848f73f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:19.902796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076df44-561d-4772-81be-db09848f73f0 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:19.903435 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076df44-561d-4772-81be-db09848f73f0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:19.903738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076df44-561d-4772-81be-db09848f73f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:19.905154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076df44-561d-4772-81be-db09848f73f0 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:19.910995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 477/959] 10.222.0.62 () {56 vars in 1122 bytes} [Thu Sep 1 16:24:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:19.917544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d89d84-9313-4820-9521-15885d8cfcb5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:20.026519 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d89d84-9313-4820-9521-15885d8cfcb5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:20.030430 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6becf-c124-44ec-ac6f-4ed0b09bef41 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:20.030709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6becf-c124-44ec-ac6f-4ed0b09bef41 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:20.030986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6becf-c124-44ec-ac6f-4ed0b09bef41 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:20.032691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 478/960] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:20.038374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25854c3-9c28-4b43-9109-e7ef9c6ca62f None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:20.038686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25854c3-9c28-4b43-9109-e7ef9c6ca62f None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:20.038930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25854c3-9c28-4b43-9109-e7ef9c6ca62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:20.059923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-f25854c3-9c28-4b43-9109-e7ef9c6ca62f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:20.061587 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f25854c3-9c28-4b43-9109-e7ef9c6ca62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:20.067684 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4a005-3a46-424b-9094-d6401b05969f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:20.068350 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4a005-3a46-424b-9094-d6401b05969f None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:20.068627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4a005-3a46-424b-9094-d6401b05969f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:20.068895 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4a005-3a46-424b-9094-d6401b05969f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:20.069580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f4a005-3a46-424b-9094-d6401b05969f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:20.071848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f4a005-3a46-424b-9094-d6401b05969f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:20.085988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:20.099996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 483/961] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:24:19 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:24:20.154768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 479/962] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:20.430254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4a005-3a46-424b-9094-d6401b05969f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:20.447377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:20.544539 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4a005-3a46-424b-9094-d6401b05969f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:20.581971 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8c13e-9d54-4b0b-871a-afe17b679494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:20.582595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8c13e-9d54-4b0b-871a-afe17b679494 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:20.583190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8c13e-9d54-4b0b-871a-afe17b679494 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:20.583454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8c13e-9d54-4b0b-871a-afe17b679494 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:20.584146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8c13e-9d54-4b0b-871a-afe17b679494 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:20.585516 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8c13e-9d54-4b0b-871a-afe17b679494 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:20.604054 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 484/963] 10.222.0.62 () {60 vars in 1141 bytes} [Thu Sep 1 16:24:20 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:24:20.913471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304458c0-65e8-464a-a055-550ebd39a10e None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:20.914024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304458c0-65e8-464a-a055-550ebd39a10e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:20.914024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304458c0-65e8-464a-a055-550ebd39a10e None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:20.915798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 480/964] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:20.921057 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4266a84-be4c-4af5-be7e-ba612a3bf49d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:20.921587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4266a84-be4c-4af5-be7e-ba612a3bf49d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:20.921587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4266a84-be4c-4af5-be7e-ba612a3bf49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:20.926017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50b6f5-af32-407b-b243-0d03f81871f1 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:20.926541 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50b6f5-af32-407b-b243-0d03f81871f1 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:20.927034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50b6f5-af32-407b-b243-0d03f81871f1 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:20.928883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 481/965] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:20.934595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728f473-153f-44ef-9037-71bed4dea235 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:20.935098 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728f473-153f-44ef-9037-71bed4dea235 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:20.935562 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728f473-153f-44ef-9037-71bed4dea235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:20.941882 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f4266a84-be4c-4af5-be7e-ba612a3bf49d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:20.943830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f4266a84-be4c-4af5-be7e-ba612a3bf49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:20.954892 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-d728f473-153f-44ef-9037-71bed4dea235 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:20.956364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d728f473-153f-44ef-9037-71bed4dea235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:21.021919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:21.034114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 485/966] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:21.047989 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 482/967] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:21.307153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4266a84-be4c-4af5-be7e-ba612a3bf49d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:21.319523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728f473-153f-44ef-9037-71bed4dea235 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:21.412558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4266a84-be4c-4af5-be7e-ba612a3bf49d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:21.423651 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:21.454340 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849b38d-27f3-497e-8350-4d79cdb0eb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:21.455344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849b38d-27f3-497e-8350-4d79cdb0eb5e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:21.455789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728f473-153f-44ef-9037-71bed4dea235 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:21.456083 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849b38d-27f3-497e-8350-4d79cdb0eb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:21.456547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849b38d-27f3-497e-8350-4d79cdb0eb5e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:21.469045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849b38d-27f3-497e-8350-4d79cdb0eb5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-d849b38d-27f3-497e-8350-4d79cdb0eb5e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:21.474592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:21.475535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:21.476440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:21.476440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:21.476440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:21.476440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:21.476440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:21.476440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:21.476440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:21.476440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:21.476440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:21.476440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:21.476440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:24:21.476440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:21.479722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 486/968] 10.222.0.62 () {56 vars in 1120 bytes} [Thu Sep 1 16:24:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:21.485806 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849b38d-27f3-497e-8350-4d79cdb0eb5e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:21.501924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52cc64-d602-45d4-b3e7-1b53651d77ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:21.503105 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52cc64-d602-45d4-b3e7-1b53651d77ca None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:21.504002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52cc64-d602-45d4-b3e7-1b53651d77ca None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:21.504445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52cc64-d602-45d4-b3e7-1b53651d77ca None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:21.505666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52cc64-d602-45d4-b3e7-1b53651d77ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:21.507915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52cc64-d602-45d4-b3e7-1b53651d77ca None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:21.527559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.sql.core [None req-fc52cc64-d602-45d4-b3e7-1b53651d77ca None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 16:24:21.527559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 16:24:21.527559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [parameters: {'user_id': 'd6cc5e5da5204b488770a58d629e9fa9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 16:24:21.527559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81523) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-fc52cc64-d602-45d4-b3e7-1b53651d77ca None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application result.read() Sep 01 16:24:21.536887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:24:21.537964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application yield resource Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise self.value Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application session.commit() Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 16:24:21.539117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.session.flush() Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application compat.raise_( Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 16:24:21.540399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:24:21.541504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application result.read() Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:24:21.544470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd6cc5e5da5204b488770a58d629e9fa9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:21.545491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 16:24:21.546555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 16:24:21.547671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 16:24:21.547671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 16:24:21.547671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 16:24:21.547671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 16:24:21.547671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 16:24:21.547671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:21.550386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 483/969] 10.222.0.62 () {60 vars in 1131 bytes} [Thu Sep 1 16:24:21 2022] POST /identity/v3/users => generated 160 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:21.557753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52cc64-d602-45d4-b3e7-1b53651d77ca None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:21.564289 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:21.611917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849b38d-27f3-497e-8350-4d79cdb0eb5e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:21.650590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48555e11-a460-43ae-b217-caf74aed295b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:21.651259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48555e11-a460-43ae-b217-caf74aed295b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:21.651618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48555e11-a460-43ae-b217-caf74aed295b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:21.651913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48555e11-a460-43ae-b217-caf74aed295b None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:21.653172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48555e11-a460-43ae-b217-caf74aed295b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:21.653172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48555e11-a460-43ae-b217-caf74aed295b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:21.654440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48555e11-a460-43ae-b217-caf74aed295b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:21.673729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 487/970] 10.222.0.62 () {56 vars in 1115 bytes} [Thu Sep 1 16:24:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:21.674792 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48555e11-a460-43ae-b217-caf74aed295b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:21.702866 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52cc64-d602-45d4-b3e7-1b53651d77ca None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:21.755174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeaa6e-902c-4c1a-aeea-6c0af9296847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:21.756242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeaa6e-902c-4c1a-aeea-6c0af9296847 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:21.756719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeaa6e-902c-4c1a-aeea-6c0af9296847 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:21.757179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeaa6e-902c-4c1a-aeea-6c0af9296847 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:21.766840 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aeaa6e-902c-4c1a-aeea-6c0af9296847 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:21.769646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aeaa6e-902c-4c1a-aeea-6c0af9296847 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:21.774188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-07aeaa6e-902c-4c1a-aeea-6c0af9296847 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:21.784560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:21.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:21.787469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:21.787469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:21.787469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:21.787469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:21.787469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:21.787469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:21.787469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:21.787469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:21.787469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:24:21.787469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:21.790992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 484/971] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:24:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:21.792506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeaa6e-902c-4c1a-aeea-6c0af9296847 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:21.803109 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48555e11-a460-43ae-b217-caf74aed295b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:21.831344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:21.850172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee019ce-0c88-4758-8352-db9483319e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:21.851420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee019ce-0c88-4758-8352-db9483319e78 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:21.851916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee019ce-0c88-4758-8352-db9483319e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:21.852725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee019ce-0c88-4758-8352-db9483319e78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:21.868427 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee019ce-0c88-4758-8352-db9483319e78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:21.870143 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee019ce-0c88-4758-8352-db9483319e78 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9ee019ce-0c88-4758-8352-db9483319e78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:21.888845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:21.889815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:21.890705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:21.890705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:21.890705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:21.897823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 488/972] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:21.898836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee019ce-0c88-4758-8352-db9483319e78 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:21.907981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeaa6e-902c-4c1a-aeea-6c0af9296847 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:21.950953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f30da-b371-405d-bb35-995daa9ee629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:21.952901 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f30da-b371-405d-bb35-995daa9ee629 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:21.953393 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f30da-b371-405d-bb35-995daa9ee629 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:21.954239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f30da-b371-405d-bb35-995daa9ee629 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:21.955537 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f30da-b371-405d-bb35-995daa9ee629 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:21.956077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f30da-b371-405d-bb35-995daa9ee629 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:21.958993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f30da-b371-405d-bb35-995daa9ee629 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:22.012116 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 485/973] 10.222.0.62 () {56 vars in 1101 bytes} [Thu Sep 1 16:24:21 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:22.018381 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee019ce-0c88-4758-8352-db9483319e78 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:22.059601 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb0abf-d059-4eb2-a497-f11a8a989280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:22.060665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb0abf-d059-4eb2-a497-f11a8a989280 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:22.061140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb0abf-d059-4eb2-a497-f11a8a989280 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:22.061602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb0abf-d059-4eb2-a497-f11a8a989280 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:22.062479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb0abf-d059-4eb2-a497-f11a8a989280 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:22.063082 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb0abf-d059-4eb2-a497-f11a8a989280 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:22.064893 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb0abf-d059-4eb2-a497-f11a8a989280 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:22.080502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 489/974] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:22.088526 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f30da-b371-405d-bb35-995daa9ee629 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:22.098840 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156ef9e-d4b9-454b-870e-692cbd7f546e None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:22.099191 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156ef9e-d4b9-454b-870e-692cbd7f546e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:22.099393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156ef9e-d4b9-454b-870e-692cbd7f546e None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:22.101403 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 490/975] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:22.106742 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36d5e2-57fa-4efa-8b29-0a134c49e3b2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:22.107187 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36d5e2-57fa-4efa-8b29-0a134c49e3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:22.107292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36d5e2-57fa-4efa-8b29-0a134c49e3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:22.126715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8f36d5e2-57fa-4efa-8b29-0a134c49e3b2 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:22.128040 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8f36d5e2-57fa-4efa-8b29-0a134c49e3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:22.161682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:22.177363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:22.219222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 491/976] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:22 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:22.234143 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f30da-b371-405d-bb35-995daa9ee629 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:22.271449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b6e66-a768-4184-ad36-99259618ab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:22.272115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b6e66-a768-4184-ad36-99259618ab0d None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:22.272387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b6e66-a768-4184-ad36-99259618ab0d None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:22.272647 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b6e66-a768-4184-ad36-99259618ab0d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:22.283212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b6e66-a768-4184-ad36-99259618ab0d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:22.284709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b6e66-a768-4184-ad36-99259618ab0d None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-251b6e66-a768-4184-ad36-99259618ab0d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:22.297310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:22.298364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:22.299781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:22.299781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:22.299781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:22.299781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:22.299781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:22.299781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:22.299781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:22.299781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:22.299781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:24:22.299781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:22.299781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 486/977] 10.222.0.62 () {56 vars in 1117 bytes} [Thu Sep 1 16:24:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:22.304617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36d5e2-57fa-4efa-8b29-0a134c49e3b2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:22.414123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36d5e2-57fa-4efa-8b29-0a134c49e3b2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:22.449395 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdf6ce-4366-40c2-8de4-0a90a2da5429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:22.450013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdf6ce-4366-40c2-8de4-0a90a2da5429 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:22.450282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdf6ce-4366-40c2-8de4-0a90a2da5429 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:22.450547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdf6ce-4366-40c2-8de4-0a90a2da5429 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:22.451285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdf6ce-4366-40c2-8de4-0a90a2da5429 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:22.451579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdf6ce-4366-40c2-8de4-0a90a2da5429 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:22.453102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdf6ce-4366-40c2-8de4-0a90a2da5429 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:22.474961 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 492/978] 10.222.0.62 () {56 vars in 1149 bytes} [Thu Sep 1 16:24:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:22.481852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b6e66-a768-4184-ad36-99259618ab0d None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:22.507196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdf6ce-4366-40c2-8de4-0a90a2da5429 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:22.530440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:22.595291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b6e66-a768-4184-ad36-99259618ab0d None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:22.614388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:22.620263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdf6ce-4366-40c2-8de4-0a90a2da5429 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:22.632896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ef604-fe09-43ac-81ae-47480b383ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:22.633504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef604-fe09-43ac-81ae-47480b383ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:22.633903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef604-fe09-43ac-81ae-47480b383ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:22.634227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef604-fe09-43ac-81ae-47480b383ce8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:22.655463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45244d-62f7-4f4a-870a-130122ceba26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:22.656136 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45244d-62f7-4f4a-870a-130122ceba26 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:22.656406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45244d-62f7-4f4a-870a-130122ceba26 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:22.656674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45244d-62f7-4f4a-870a-130122ceba26 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:22.657247 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45244d-62f7-4f4a-870a-130122ceba26 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:22.658604 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45244d-62f7-4f4a-870a-130122ceba26 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-5d45244d-62f7-4f4a-870a-130122ceba26 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:22.669347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:22.670347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:22.671242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 493/979] 10.222.0.62 () {56 vars in 1126 bytes} [Thu Sep 1 16:24:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:22.672652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf73787-69b9-4093-ac09-76fd7edd39c5 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:22.672652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf73787-69b9-4093-ac09-76fd7edd39c5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:22.672834 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf73787-69b9-4093-ac09-76fd7edd39c5 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:22.674188 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 494/980] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:22.677131 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf73787-69b9-4093-ac09-76fd7edd39c5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:22.704761 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ef604-fe09-43ac-81ae-47480b383ce8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:22.706365 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ef604-fe09-43ac-81ae-47480b383ce8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:22.765483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:22.782260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf73787-69b9-4093-ac09-76fd7edd39c5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:22.804838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 487/981] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:22 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:24:22.804838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a470194-6f8f-4b25-9dac-972022c01033 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:22.804838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a470194-6f8f-4b25-9dac-972022c01033 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:22.804838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a470194-6f8f-4b25-9dac-972022c01033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:22.820194 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26f91f-935f-4e19-aafb-d6bc597d0ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:22.821004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26f91f-935f-4e19-aafb-d6bc597d0ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:22.821497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26f91f-935f-4e19-aafb-d6bc597d0ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:22.822317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26f91f-935f-4e19-aafb-d6bc597d0ce5 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:22.822960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26f91f-935f-4e19-aafb-d6bc597d0ce5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:22.823279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26f91f-935f-4e19-aafb-d6bc597d0ce5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:22.824657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26f91f-935f-4e19-aafb-d6bc597d0ce5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:22.824849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-0a470194-6f8f-4b25-9dac-972022c01033 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:22.826346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0a470194-6f8f-4b25-9dac-972022c01033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:22.829765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 495/982] 10.222.0.62 () {56 vars in 1122 bytes} [Thu Sep 1 16:24:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:22.830537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26f91f-935f-4e19-aafb-d6bc597d0ce5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:22.840909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:22.896503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 488/983] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:22 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:22.897356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a470194-6f8f-4b25-9dac-972022c01033 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:22.945945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26f91f-935f-4e19-aafb-d6bc597d0ce5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:22.983011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1fba9-5d4c-42fa-b87a-6269b4ee92d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:22.983650 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1fba9-5d4c-42fa-b87a-6269b4ee92d5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:22.983954 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1fba9-5d4c-42fa-b87a-6269b4ee92d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:22.984226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1fba9-5d4c-42fa-b87a-6269b4ee92d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:22.995679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1fba9-5d4c-42fa-b87a-6269b4ee92d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:22.997150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1fba9-5d4c-42fa-b87a-6269b4ee92d5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:23.003719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a470194-6f8f-4b25-9dac-972022c01033 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:23.006822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-5ba1fba9-5d4c-42fa-b87a-6269b4ee92d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:23.014623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:23.015825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:23.016966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:23.016966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:23.016966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:23.019084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 496/984] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:23.024954 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1fba9-5d4c-42fa-b87a-6269b4ee92d5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:23.042888 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061ffff-a5b1-4eea-834d-d585129ea6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:23.043795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061ffff-a5b1-4eea-834d-d585129ea6ac None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:23.044278 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061ffff-a5b1-4eea-834d-d585129ea6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:23.044737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061ffff-a5b1-4eea-834d-d585129ea6ac None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:23.045447 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061ffff-a5b1-4eea-834d-d585129ea6ac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:23.045813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061ffff-a5b1-4eea-834d-d585129ea6ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:23.047250 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061ffff-a5b1-4eea-834d-d585129ea6ac None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:23.057124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 489/985] 10.222.0.62 () {56 vars in 1122 bytes} [Thu Sep 1 16:24:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:23.058145 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061ffff-a5b1-4eea-834d-d585129ea6ac None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:23.073368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:23.133510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1fba9-5d4c-42fa-b87a-6269b4ee92d5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:23.163785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061ffff-a5b1-4eea-834d-d585129ea6ac None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:23.168568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94473b-2f64-4240-a9ff-4a901a601ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:23.169248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94473b-2f64-4240-a9ff-4a901a601ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:23.169593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94473b-2f64-4240-a9ff-4a901a601ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:23.169961 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94473b-2f64-4240-a9ff-4a901a601ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:23.170666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94473b-2f64-4240-a9ff-4a901a601ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:23.171032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94473b-2f64-4240-a9ff-4a901a601ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:23.172505 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94473b-2f64-4240-a9ff-4a901a601ea5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:23.185095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 497/986] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:23.185975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94473b-2f64-4240-a9ff-4a901a601ea5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:23.201850 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb4b4a-43cf-48e9-8773-09c6804de7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:23.202390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb4b4a-43cf-48e9-8773-09c6804de7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:23.202691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb4b4a-43cf-48e9-8773-09c6804de7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:23.202935 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb4b4a-43cf-48e9-8773-09c6804de7d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:23.203639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb4b4a-43cf-48e9-8773-09c6804de7d1 None admin] RBAC: Adding query filter params (interface=admin, service_id=cbf823fd81844e91b44cbf4b43a53719, region_id=RegionOne) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:23.203972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb4b4a-43cf-48e9-8773-09c6804de7d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:23.205337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb4b4a-43cf-48e9-8773-09c6804de7d1 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:23.210803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 490/987] 10.222.0.62 () {56 vars in 1249 bytes} [Thu Sep 1 16:24:23 2022] GET /identity/v3/endpoints?service_id=cbf823fd81844e91b44cbf4b43a53719&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:23.211403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb4b4a-43cf-48e9-8773-09c6804de7d1 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:23.249710 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:23.291493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:23.292139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94473b-2f64-4240-a9ff-4a901a601ea5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:23.316287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb4b4a-43cf-48e9-8773-09c6804de7d1 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:23.328279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548fec9-8b74-4234-b2d9-21c8d0e8c323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:23.328997 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548fec9-8b74-4234-b2d9-21c8d0e8c323 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:23.329308 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548fec9-8b74-4234-b2d9-21c8d0e8c323 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:23.329574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548fec9-8b74-4234-b2d9-21c8d0e8c323 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:23.330154 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548fec9-8b74-4234-b2d9-21c8d0e8c323 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:23.331561 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548fec9-8b74-4234-b2d9-21c8d0e8c323 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-1548fec9-8b74-4234-b2d9-21c8d0e8c323 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:23.341172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:23.342430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:23.343281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 498/988] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:23.344121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548fec9-8b74-4234-b2d9-21c8d0e8c323 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:23.353211 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20476c3-9ae6-46c5-b8c5-e0dea0a604a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:23.353846 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20476c3-9ae6-46c5-b8c5-e0dea0a604a4 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:23.354114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20476c3-9ae6-46c5-b8c5-e0dea0a604a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:23.354380 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20476c3-9ae6-46c5-b8c5-e0dea0a604a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:23.361060 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20476c3-9ae6-46c5-b8c5-e0dea0a604a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:23.362491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20476c3-9ae6-46c5-b8c5-e0dea0a604a4 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-a20476c3-9ae6-46c5-b8c5-e0dea0a604a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:23.374576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:23.375626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:23.376754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:23.376754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:23.376754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:23.376754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:23.376754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:23.376754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:23.376754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:23.376754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:23.376754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 491/989] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:23.379719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20476c3-9ae6-46c5-b8c5-e0dea0a604a4 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:23.445111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548fec9-8b74-4234-b2d9-21c8d0e8c323 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:23.470209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:23.479200 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20476c3-9ae6-46c5-b8c5-e0dea0a604a4 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:23.481961 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43ef64-e3a0-4998-9112-87ad0ce4d4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:23.482564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43ef64-e3a0-4998-9112-87ad0ce4d4fa None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:23.482834 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43ef64-e3a0-4998-9112-87ad0ce4d4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:23.483097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43ef64-e3a0-4998-9112-87ad0ce4d4fa None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:23.483694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43ef64-e3a0-4998-9112-87ad0ce4d4fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:23.485088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43ef64-e3a0-4998-9112-87ad0ce4d4fa None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:23.491163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 499/990] 10.222.0.62 () {56 vars in 1087 bytes} [Thu Sep 1 16:24:23 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:24:23.491933 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43ef64-e3a0-4998-9112-87ad0ce4d4fa None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:23.516407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9129d-3518-40a6-996a-1f4ead262491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:23.517081 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9129d-3518-40a6-996a-1f4ead262491 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:23.517443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9129d-3518-40a6-996a-1f4ead262491 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:23.517773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9129d-3518-40a6-996a-1f4ead262491 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:23.518449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9129d-3518-40a6-996a-1f4ead262491 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:23.518747 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9129d-3518-40a6-996a-1f4ead262491 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:23.520244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9129d-3518-40a6-996a-1f4ead262491 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:23.523025 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:23.532582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 492/991] 10.222.0.62 () {56 vars in 1112 bytes} [Thu Sep 1 16:24:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:23.533616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9129d-3518-40a6-996a-1f4ead262491 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:23.617612 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43ef64-e3a0-4998-9112-87ad0ce4d4fa None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:23.644359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9129d-3518-40a6-996a-1f4ead262491 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:23.658581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1ad19-9331-49ee-8f90-afa4e54c246a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:23.659192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1ad19-9331-49ee-8f90-afa4e54c246a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:23.659464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1ad19-9331-49ee-8f90-afa4e54c246a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:23.659733 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1ad19-9331-49ee-8f90-afa4e54c246a None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:23.660463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1ad19-9331-49ee-8f90-afa4e54c246a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:23.662222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1ad19-9331-49ee-8f90-afa4e54c246a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:23.662222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1ad19-9331-49ee-8f90-afa4e54c246a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:23.675284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 500/992] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:23.682010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1ad19-9331-49ee-8f90-afa4e54c246a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:23.686483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e9413-cc5c-4480-9311-8d1483443691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:23.687106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e9413-cc5c-4480-9311-8d1483443691 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:23.687383 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e9413-cc5c-4480-9311-8d1483443691 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:23.687649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e9413-cc5c-4480-9311-8d1483443691 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:23.688329 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e9413-cc5c-4480-9311-8d1483443691 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:23.688632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e9413-cc5c-4480-9311-8d1483443691 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:23.690005 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e9413-cc5c-4480-9311-8d1483443691 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:23.699957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 493/993] 10.222.0.62 () {56 vars in 1112 bytes} [Thu Sep 1 16:24:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:23.701541 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e9413-cc5c-4480-9311-8d1483443691 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:23.737046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:23.775905 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:23.792079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1ad19-9331-49ee-8f90-afa4e54c246a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:23.823924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e9413-cc5c-4480-9311-8d1483443691 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:23.826317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72dea25-0466-4e38-8c7e-129af9ed2089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:23.826995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72dea25-0466-4e38-8c7e-129af9ed2089 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:23.827339 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72dea25-0466-4e38-8c7e-129af9ed2089 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:23.827671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72dea25-0466-4e38-8c7e-129af9ed2089 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:23.828502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72dea25-0466-4e38-8c7e-129af9ed2089 None admin] RBAC: Adding query filter params (role.id=814881b0c50648f4a0edcdf280dd29c7, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=24705faaf5054802a556749c7ba82ee8) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:23.828874 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72dea25-0466-4e38-8c7e-129af9ed2089 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:23.830358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72dea25-0466-4e38-8c7e-129af9ed2089 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:23.837292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 501/994] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:23 2022] GET /identity/v3/role_assignments?user.id=24705faaf5054802a556749c7ba82ee8&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=814881b0c50648f4a0edcdf280dd29c7 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:23.860890 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32288308-02b4-4110-8c6c-12d3d552a9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:23.861525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32288308-02b4-4110-8c6c-12d3d552a9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:23.861796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32288308-02b4-4110-8c6c-12d3d552a9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:23.862058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32288308-02b4-4110-8c6c-12d3d552a9a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:23.862776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32288308-02b4-4110-8c6c-12d3d552a9a9 None admin] RBAC: Adding query filter params (interface=public, service_id=184f001817644c1d88450c8986f3e3af, region_id=RegionOne) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:23.863077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32288308-02b4-4110-8c6c-12d3d552a9a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:23.864549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32288308-02b4-4110-8c6c-12d3d552a9a9 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:23.875402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 494/995] 10.222.0.62 () {56 vars in 1251 bytes} [Thu Sep 1 16:24:23 2022] GET /identity/v3/endpoints?service_id=184f001817644c1d88450c8986f3e3af&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:23.877062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72dea25-0466-4e38-8c7e-129af9ed2089 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:23.987859 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72dea25-0466-4e38-8c7e-129af9ed2089 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:24.002643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:24.025959 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da3dd4-77b1-4c22-abe2-e73416237d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:24.026592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da3dd4-77b1-4c22-abe2-e73416237d84 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:24.026829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da3dd4-77b1-4c22-abe2-e73416237d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:24.027095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da3dd4-77b1-4c22-abe2-e73416237d84 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:24.027723 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da3dd4-77b1-4c22-abe2-e73416237d84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:24.029321 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da3dd4-77b1-4c22-abe2-e73416237d84 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:24.036215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 502/996] 10.222.0.62 () {56 vars in 1087 bytes} [Thu Sep 1 16:24:23 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:24:25.381944 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73e975-846f-4cea-9a55-26ea34af034b None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:25.382493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73e975-846f-4cea-9a55-26ea34af034b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:25.382493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73e975-846f-4cea-9a55-26ea34af034b None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:25.383921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 495/997] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:25.389084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf0ae5-d2bd-4964-981b-114f15734252 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:25.389506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf0ae5-d2bd-4964-981b-114f15734252 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:25.389582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf0ae5-d2bd-4964-981b-114f15734252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:25.408078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-22cf0ae5-d2bd-4964-981b-114f15734252 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:25.409377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-22cf0ae5-d2bd-4964-981b-114f15734252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:25.480625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 503/998] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:25.740642 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73e975-846f-4cea-9a55-26ea34af034b None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:25.786659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69a4d7-15b7-44ca-8d86-4c619510f9a3 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:25.786998 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69a4d7-15b7-44ca-8d86-4c619510f9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:25.787211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69a4d7-15b7-44ca-8d86-4c619510f9a3 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:25.788717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 504/999] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:25.794479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcce179-1b92-4f95-844f-ddfee6d0f5a7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:25.794891 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcce179-1b92-4f95-844f-ddfee6d0f5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:25.795293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcce179-1b92-4f95-844f-ddfee6d0f5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:25.815524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6dcce179-1b92-4f95-844f-ddfee6d0f5a7 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:25.816890 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6dcce179-1b92-4f95-844f-ddfee6d0f5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:25.854641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73e975-846f-4cea-9a55-26ea34af034b None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:25.881927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:25.892017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bacc24-eeea-4db9-83c0-f84426291c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:25.892655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bacc24-eeea-4db9-83c0-f84426291c95 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:25.892929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bacc24-eeea-4db9-83c0-f84426291c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:25.893193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bacc24-eeea-4db9-83c0-f84426291c95 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:25.893791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bacc24-eeea-4db9-83c0-f84426291c95 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:25.895152 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bacc24-eeea-4db9-83c0-f84426291c95 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:25.903074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 505/1000] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-b0bacc24-eeea-4db9-83c0-f84426291c95 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:25.904670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:25.905758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:25.906582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:25.906582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:25.906582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:25.906582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:25.906582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:25.906582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:25.906582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:25.906582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:25.906582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:25.906582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:25.906582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:25.906582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:25.906582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:24:25.906582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:25.910910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 496/1001] 10.222.0.62 () {56 vars in 1126 bytes} [Thu Sep 1 16:24:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:25.911915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcce179-1b92-4f95-844f-ddfee6d0f5a7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:25.919046 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f41f08-dcb2-4b95-b520-2146f128eff6 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:25.919328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f41f08-dcb2-4b95-b520-2146f128eff6 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:25.919596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f41f08-dcb2-4b95-b520-2146f128eff6 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:25.923424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 497/1002] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:25.925635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a47da6-18f4-4663-a1af-f543ac96c724 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:25.925912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a47da6-18f4-4663-a1af-f543ac96c724 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:25.926178 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a47da6-18f4-4663-a1af-f543ac96c724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:25.944958 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-80a47da6-18f4-4663-a1af-f543ac96c724 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:25.946019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-80a47da6-18f4-4663-a1af-f543ac96c724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:25.949051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:26.016878 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 498/1003] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:26.021866 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcce179-1b92-4f95-844f-ddfee6d0f5a7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:26.056175 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b16a4-c673-4db6-94ee-52f17fc55b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:26.056811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b16a4-c673-4db6-94ee-52f17fc55b68 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:26.057086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b16a4-c673-4db6-94ee-52f17fc55b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:26.057350 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b16a4-c673-4db6-94ee-52f17fc55b68 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:26.057973 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b16a4-c673-4db6-94ee-52f17fc55b68 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:26.058268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b16a4-c673-4db6-94ee-52f17fc55b68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:26.059681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b16a4-c673-4db6-94ee-52f17fc55b68 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:26.069101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 506/1004] 10.222.0.62 () {56 vars in 1122 bytes} [Thu Sep 1 16:24:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:26.070679 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a47da6-18f4-4663-a1af-f543ac96c724 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:26.164305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b16a4-c673-4db6-94ee-52f17fc55b68 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:26.181657 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a47da6-18f4-4663-a1af-f543ac96c724 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:26.208360 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:26.221324 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ec56b-8e22-4086-8c5d-832fd15e653e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:26.221994 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ec56b-8e22-4086-8c5d-832fd15e653e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:26.222201 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ec56b-8e22-4086-8c5d-832fd15e653e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:26.222464 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ec56b-8e22-4086-8c5d-832fd15e653e None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:26.223090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ec56b-8e22-4086-8c5d-832fd15e653e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:26.223387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ec56b-8e22-4086-8c5d-832fd15e653e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:26.224849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ec56b-8e22-4086-8c5d-832fd15e653e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:26.235067 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 499/1005] 10.222.0.62 () {56 vars in 1122 bytes} [Thu Sep 1 16:24:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:26.236069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ec56b-8e22-4086-8c5d-832fd15e653e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:26.278227 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:26.281969 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b16a4-c673-4db6-94ee-52f17fc55b68 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:26.317341 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4448d-8e94-412d-be7e-d05e18e461ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:26.317956 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4448d-8e94-412d-be7e-d05e18e461ad None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:26.318231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4448d-8e94-412d-be7e-d05e18e461ad None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:26.318499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4448d-8e94-412d-be7e-d05e18e461ad None admin] PATH_INFO: `/v3/services/image` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:26.319076 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4448d-8e94-412d-be7e-d05e18e461ad None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:26.320456 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4448d-8e94-412d-be7e-d05e18e461ad None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-04c4448d-8e94-412d-be7e-d05e18e461ad None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:26.329077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:26.330244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:26.331080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 507/1006] 10.222.0.62 () {56 vars in 1105 bytes} [Thu Sep 1 16:24:26 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:26.332046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4448d-8e94-412d-be7e-d05e18e461ad None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:26.342157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ec56b-8e22-4086-8c5d-832fd15e653e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:26.375503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051ee68-58e3-4c1d-a88e-7ae43240da9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:26.376141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051ee68-58e3-4c1d-a88e-7ae43240da9f None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:26.376415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051ee68-58e3-4c1d-a88e-7ae43240da9f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:26.376752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051ee68-58e3-4c1d-a88e-7ae43240da9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:26.377417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051ee68-58e3-4c1d-a88e-7ae43240da9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:26.378781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051ee68-58e3-4c1d-a88e-7ae43240da9f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:26.395991 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 500/1007] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:24:26 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:24:26.396794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051ee68-58e3-4c1d-a88e-7ae43240da9f None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:26.447092 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4448d-8e94-412d-be7e-d05e18e461ad None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:26.449488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:26.492415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d08cb-fc4d-4999-9b45-83b55ce17d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:26.493063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d08cb-fc4d-4999-9b45-83b55ce17d74 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:26.493406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d08cb-fc4d-4999-9b45-83b55ce17d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:26.493729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d08cb-fc4d-4999-9b45-83b55ce17d74 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:26.494319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d08cb-fc4d-4999-9b45-83b55ce17d74 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:26.495698 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d08cb-fc4d-4999-9b45-83b55ce17d74 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-862d08cb-fc4d-4999-9b45-83b55ce17d74 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:26.505272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:26.506394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:26.507346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 508/1008] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:26.513085 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d08cb-fc4d-4999-9b45-83b55ce17d74 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:26.521717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051ee68-58e3-4c1d-a88e-7ae43240da9f None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:26.547785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:26.560611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571154a5-e81e-44fc-a55c-17d02976a0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:26.561561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571154a5-e81e-44fc-a55c-17d02976a0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:26.561896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571154a5-e81e-44fc-a55c-17d02976a0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:26.562233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571154a5-e81e-44fc-a55c-17d02976a0b1 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:26.562870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571154a5-e81e-44fc-a55c-17d02976a0b1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:26.563168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571154a5-e81e-44fc-a55c-17d02976a0b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:26.564683 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571154a5-e81e-44fc-a55c-17d02976a0b1 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:26.569672 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 501/1009] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:26.575690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571154a5-e81e-44fc-a55c-17d02976a0b1 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:26.634637 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d08cb-fc4d-4999-9b45-83b55ce17d74 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:26.670858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae284c4-ff11-4cb6-b457-49bf79dbbc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:26.671963 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae284c4-ff11-4cb6-b457-49bf79dbbc1b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:26.672540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae284c4-ff11-4cb6-b457-49bf79dbbc1b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:26.672983 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae284c4-ff11-4cb6-b457-49bf79dbbc1b None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:26.673767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae284c4-ff11-4cb6-b457-49bf79dbbc1b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:26.674211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae284c4-ff11-4cb6-b457-49bf79dbbc1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:26.675848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae284c4-ff11-4cb6-b457-49bf79dbbc1b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:26.681729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 509/1010] 10.222.0.62 () {56 vars in 1112 bytes} [Thu Sep 1 16:24:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:26.687315 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae284c4-ff11-4cb6-b457-49bf79dbbc1b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:26.689808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571154a5-e81e-44fc-a55c-17d02976a0b1 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:26.700771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:26.726939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcd94d-ea11-40ff-a318-14cde6fec602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:26.727609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcd94d-ea11-40ff-a318-14cde6fec602 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:26.727950 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcd94d-ea11-40ff-a318-14cde6fec602 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:26.728228 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcd94d-ea11-40ff-a318-14cde6fec602 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:26.728852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcd94d-ea11-40ff-a318-14cde6fec602 None admin] RBAC: Adding query filter params (type=image) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:26.729154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcd94d-ea11-40ff-a318-14cde6fec602 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:26.730561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcd94d-ea11-40ff-a318-14cde6fec602 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:26.744034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 502/1011] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:26 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:26.794611 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:26.809460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae284c4-ff11-4cb6-b457-49bf79dbbc1b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:26.847220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006096a1-b66b-4e77-acb0-122350d9be30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:26.847847 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006096a1-b66b-4e77-acb0-122350d9be30 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:26.848158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006096a1-b66b-4e77-acb0-122350d9be30 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:26.848431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006096a1-b66b-4e77-acb0-122350d9be30 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:26.849079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006096a1-b66b-4e77-acb0-122350d9be30 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:26.849380 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006096a1-b66b-4e77-acb0-122350d9be30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:26.850771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006096a1-b66b-4e77-acb0-122350d9be30 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:26.857028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 510/1012] 10.222.0.62 () {56 vars in 1112 bytes} [Thu Sep 1 16:24:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:26.862849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcd94d-ea11-40ff-a318-14cde6fec602 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:26.967678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcd94d-ea11-40ff-a318-14cde6fec602 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:27.003281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0984539-0eaa-4944-9f39-103c8d4fa4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:27.003910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0984539-0eaa-4944-9f39-103c8d4fa4b0 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:27.004194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0984539-0eaa-4944-9f39-103c8d4fa4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:27.004467 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0984539-0eaa-4944-9f39-103c8d4fa4b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:27.005150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0984539-0eaa-4944-9f39-103c8d4fa4b0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:27.006522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0984539-0eaa-4944-9f39-103c8d4fa4b0 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:27.029107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 503/1013] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:24:26 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:24:28.302128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e057f3-4325-4f47-af70-8b040b911d99 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:28.302725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e057f3-4325-4f47-af70-8b040b911d99 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:28.302725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e057f3-4325-4f47-af70-8b040b911d99 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:28.304156 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 511/1014] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:28.309357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd144c8-8ff0-4bac-9839-93ad9c737d0e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:28.309810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd144c8-8ff0-4bac-9839-93ad9c737d0e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:28.309891 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd144c8-8ff0-4bac-9839-93ad9c737d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:28.328547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-9cd144c8-8ff0-4bac-9839-93ad9c737d0e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:28.329817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9cd144c8-8ff0-4bac-9839-93ad9c737d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:28.355272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:28.407731 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 504/1015] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:28 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:28.659218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271b7e-f2cb-4815-ab67-3c6aa1d09ae1 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:28.659623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271b7e-f2cb-4815-ab67-3c6aa1d09ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:28.659725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271b7e-f2cb-4815-ab67-3c6aa1d09ae1 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:28.661272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 512/1016] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:28.666699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a163f-b7d0-47e4-bf67-9dd8fde99dd9 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:28.666951 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a163f-b7d0-47e4-bf67-9dd8fde99dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:28.667253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a163f-b7d0-47e4-bf67-9dd8fde99dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:28.675647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5271b7e-f2cb-4815-ab67-3c6aa1d09ae1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:28.691907 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-cd3a163f-b7d0-47e4-bf67-9dd8fde99dd9 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:28.693095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-cd3a163f-b7d0-47e4-bf67-9dd8fde99dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:28.777011 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 505/1017] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:28 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:28.785919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5271b7e-f2cb-4815-ab67-3c6aa1d09ae1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:28.821121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cac13a-9623-4bea-a4a0-b2e7393e10ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:28.821870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cac13a-9623-4bea-a4a0-b2e7393e10ba None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:28.822144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cac13a-9623-4bea-a4a0-b2e7393e10ba None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:28.822410 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cac13a-9623-4bea-a4a0-b2e7393e10ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:28.823515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cac13a-9623-4bea-a4a0-b2e7393e10ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:28.824968 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cac13a-9623-4bea-a4a0-b2e7393e10ba None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:28.828207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 513/1018] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:28.832292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a163f-b7d0-47e4-bf67-9dd8fde99dd9 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:28.920941 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:28.937111 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a163f-b7d0-47e4-bf67-9dd8fde99dd9 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:28.970495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8457057-bd69-4a6a-9ec9-5a644b8741c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:28.971102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8457057-bd69-4a6a-9ec9-5a644b8741c3 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:28.971372 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8457057-bd69-4a6a-9ec9-5a644b8741c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:28.971635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8457057-bd69-4a6a-9ec9-5a644b8741c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:28.972308 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8457057-bd69-4a6a-9ec9-5a644b8741c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:28.973741 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8457057-bd69-4a6a-9ec9-5a644b8741c3 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:29.003921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 506/1019] 10.222.0.62 () {60 vars in 1141 bytes} [Thu Sep 1 16:24:28 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:24:29.005750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f65a09-799d-4ec4-a6b6-78e7ee58921f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:29.006032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f65a09-799d-4ec4-a6b6-78e7ee58921f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:29.006307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f65a09-799d-4ec4-a6b6-78e7ee58921f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:29.007927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 514/1020] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:29.013873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b7ac0-e58a-4fd7-a968-d5cf499f9095 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:29.014170 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b7ac0-e58a-4fd7-a968-d5cf499f9095 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:29.014567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b7ac0-e58a-4fd7-a968-d5cf499f9095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:29.034171 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-501b7ac0-e58a-4fd7-a968-d5cf499f9095 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:29.035412 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-501b7ac0-e58a-4fd7-a968-d5cf499f9095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:29.046503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f65a09-799d-4ec4-a6b6-78e7ee58921f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:29.070762 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:29.113727 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 507/1021] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:29 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:29.170075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f65a09-799d-4ec4-a6b6-78e7ee58921f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:29.208912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb109555-ab03-49d2-8f08-8eebb495056b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:29.211268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb109555-ab03-49d2-8f08-8eebb495056b None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:29.211268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb109555-ab03-49d2-8f08-8eebb495056b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:29.211268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb109555-ab03-49d2-8f08-8eebb495056b None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:29.211268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb109555-ab03-49d2-8f08-8eebb495056b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:29.212069 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb109555-ab03-49d2-8f08-8eebb495056b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:29.229542 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 515/1022] 10.222.0.62 () {60 vars in 1141 bytes} [Thu Sep 1 16:24:29 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:24:29.379594 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b7ac0-e58a-4fd7-a968-d5cf499f9095 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:29.448789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:29.485876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b7ac0-e58a-4fd7-a968-d5cf499f9095 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:29.531507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79984d21-e4fd-47eb-8a5b-daf17ce5005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:29.532179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984d21-e4fd-47eb-8a5b-daf17ce5005e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:29.532455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984d21-e4fd-47eb-8a5b-daf17ce5005e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:29.532721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79984d21-e4fd-47eb-8a5b-daf17ce5005e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:29.543547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79984d21-e4fd-47eb-8a5b-daf17ce5005e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-79984d21-e4fd-47eb-8a5b-daf17ce5005e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:29.548452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:29.549374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:29.550746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:29.550746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:29.550746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:29.550746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:29.550746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:29.550746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:29.550746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:29.550746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:29.550746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:29.550746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:29.550746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:24:29.550746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:29.550746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 508/1023] 10.222.0.62 () {56 vars in 1120 bytes} [Thu Sep 1 16:24:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:29.555715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb109555-ab03-49d2-8f08-8eebb495056b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:29.644943 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:29.669266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb109555-ab03-49d2-8f08-8eebb495056b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:29.705464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f824ef5-d398-4b43-947e-3f7f960b1655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:29.706327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f824ef5-d398-4b43-947e-3f7f960b1655 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:29.706601 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f824ef5-d398-4b43-947e-3f7f960b1655 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:29.706865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f824ef5-d398-4b43-947e-3f7f960b1655 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:29.707520 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f824ef5-d398-4b43-947e-3f7f960b1655 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:29.708159 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f824ef5-d398-4b43-947e-3f7f960b1655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:29.709511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f824ef5-d398-4b43-947e-3f7f960b1655 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:29.718173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 516/1024] 10.222.0.62 () {56 vars in 1115 bytes} [Thu Sep 1 16:24:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:29.724801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79984d21-e4fd-47eb-8a5b-daf17ce5005e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:29.834874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79984d21-e4fd-47eb-8a5b-daf17ce5005e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:29.858121 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:29.870083 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90747283-86ef-420f-949f-473fda3697df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:29.870712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747283-86ef-420f-949f-473fda3697df None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:29.870985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747283-86ef-420f-949f-473fda3697df None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:29.871250 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747283-86ef-420f-949f-473fda3697df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:29.881395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90747283-86ef-420f-949f-473fda3697df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:29.882874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90747283-86ef-420f-949f-473fda3697df None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-90747283-86ef-420f-949f-473fda3697df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:29.898216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:29.899197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:29.905141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:29.905141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:29.905141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:29.905141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 509/1025] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:29.907225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f824ef5-d398-4b43-947e-3f7f960b1655 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:30.014972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f824ef5-d398-4b43-947e-3f7f960b1655 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:30.048440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8d6ce-c820-43b5-8319-978e388a6d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:30.049043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8d6ce-c820-43b5-8319-978e388a6d25 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:30.049313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8d6ce-c820-43b5-8319-978e388a6d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:30.049574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8d6ce-c820-43b5-8319-978e388a6d25 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:30.050202 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8d6ce-c820-43b5-8319-978e388a6d25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:30.050494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8d6ce-c820-43b5-8319-978e388a6d25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:30.051840 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8d6ce-c820-43b5-8319-978e388a6d25 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:30.068656 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 517/1026] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:30.070263 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90747283-86ef-420f-949f-473fda3697df None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:30.174997 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90747283-86ef-420f-949f-473fda3697df None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:30.208427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d019f76-bd2f-4eb0-bbe1-5ae4c38a7ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:30.209051 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d019f76-bd2f-4eb0-bbe1-5ae4c38a7ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:30.209318 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d019f76-bd2f-4eb0-bbe1-5ae4c38a7ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:30.209583 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d019f76-bd2f-4eb0-bbe1-5ae4c38a7ed4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:30.217972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d019f76-bd2f-4eb0-bbe1-5ae4c38a7ed4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:30.219649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d019f76-bd2f-4eb0-bbe1-5ae4c38a7ed4 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-5d019f76-bd2f-4eb0-bbe1-5ae4c38a7ed4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:30.234371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:30.235523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:30.236694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:30.236694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:30.236694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:30.236694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:30.236694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:30.236694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:30.236694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:30.236694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:30.236694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:24:30.236694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:30.236694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 510/1027] 10.222.0.62 () {56 vars in 1093 bytes} [Thu Sep 1 16:24:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:30.241618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8d6ce-c820-43b5-8319-978e388a6d25 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:30.245045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:30.350438 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8d6ce-c820-43b5-8319-978e388a6d25 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:30.385185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82946d0-c7c2-4172-9636-29171f6c4c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:30.385809 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82946d0-c7c2-4172-9636-29171f6c4c07 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:30.386079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82946d0-c7c2-4172-9636-29171f6c4c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:30.386345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82946d0-c7c2-4172-9636-29171f6c4c07 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:30.387048 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82946d0-c7c2-4172-9636-29171f6c4c07 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:30.387350 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82946d0-c7c2-4172-9636-29171f6c4c07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:30.388790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82946d0-c7c2-4172-9636-29171f6c4c07 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:30.406221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 518/1028] 10.222.0.62 () {56 vars in 1133 bytes} [Thu Sep 1 16:24:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:30.412069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d019f76-bd2f-4eb0-bbe1-5ae4c38a7ed4 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:30.451130 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:30.519564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d019f76-bd2f-4eb0-bbe1-5ae4c38a7ed4 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:30.557029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0254320-7fde-4952-9040-be51aeb90226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:30.557652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0254320-7fde-4952-9040-be51aeb90226 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:30.557910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0254320-7fde-4952-9040-be51aeb90226 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:30.558202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0254320-7fde-4952-9040-be51aeb90226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:30.630168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0254320-7fde-4952-9040-be51aeb90226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:30.631582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0254320-7fde-4952-9040-be51aeb90226 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:30.670460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:30.740828 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 511/1029] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:30 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:24:30.746953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82946d0-c7c2-4172-9636-29171f6c4c07 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:30.817515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:30.869929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82946d0-c7c2-4172-9636-29171f6c4c07 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:30.916928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df9e8d-3667-43b9-9e0f-b7bda05cf9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:30.917777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df9e8d-3667-43b9-9e0f-b7bda05cf9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:30.918195 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df9e8d-3667-43b9-9e0f-b7bda05cf9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:30.918596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df9e8d-3667-43b9-9e0f-b7bda05cf9b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:30.934095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df9e8d-3667-43b9-9e0f-b7bda05cf9b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:30.936404 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df9e8d-3667-43b9-9e0f-b7bda05cf9b4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:30.940692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61834-85fd-4546-afa4-66c5e3e25f5d None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:30.940973 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61834-85fd-4546-afa4-66c5e3e25f5d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:30.941241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e61834-85fd-4546-afa4-66c5e3e25f5d None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:30.942726 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 512/1030] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:30.947947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8dbbf4-e891-4533-9376-4bbd30476238 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:30.948225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8dbbf4-e891-4533-9376-4bbd30476238 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:30.948494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8dbbf4-e891-4533-9376-4bbd30476238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-28df9e8d-3667-43b9-9e0f-b7bda05cf9b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:30.957110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:30.958379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:30.959274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:30.959274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:30.959274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:30.964279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 519/1031] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:30.965666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df9e8d-3667-43b9-9e0f-b7bda05cf9b4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:30.968313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-4b8dbbf4-e891-4533-9376-4bbd30476238 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:30.969538 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-4b8dbbf4-e891-4533-9376-4bbd30476238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:31.041267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 513/1032] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:31.074702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df9e8d-3667-43b9-9e0f-b7bda05cf9b4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:31.088576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:31.111497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e31c7-21e9-46e8-b4ee-e5f7b716b17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:31.112136 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e31c7-21e9-46e8-b4ee-e5f7b716b17b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.112409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e31c7-21e9-46e8-b4ee-e5f7b716b17b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.112676 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e31c7-21e9-46e8-b4ee-e5f7b716b17b None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.113307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e31c7-21e9-46e8-b4ee-e5f7b716b17b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:31.113606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e31c7-21e9-46e8-b4ee-e5f7b716b17b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:31.115008 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e31c7-21e9-46e8-b4ee-e5f7b716b17b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:31.127215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 520/1033] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:31.132742 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8dbbf4-e891-4533-9376-4bbd30476238 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:31.169092 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4eb98-2870-4bfb-9e00-a8f139ad4277 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.169285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4eb98-2870-4bfb-9e00-a8f139ad4277 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.169541 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4eb98-2870-4bfb-9e00-a8f139ad4277 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.170863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 521/1034] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:31.176197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ad55f-67f1-4950-9a94-381c16b796be None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.176738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ad55f-67f1-4950-9a94-381c16b796be None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.176738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ad55f-67f1-4950-9a94-381c16b796be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.195640 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f08ad55f-67f1-4950-9a94-381c16b796be None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:31.197278 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f08ad55f-67f1-4950-9a94-381c16b796be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:31.201987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:31.243676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8dbbf4-e891-4533-9376-4bbd30476238 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:31.278077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 522/1035] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:31.282823 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f13b2-d188-4795-ac05-36ebe888fb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:31.283469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f13b2-d188-4795-ac05-36ebe888fb26 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.283742 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f13b2-d188-4795-ac05-36ebe888fb26 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.284091 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f13b2-d188-4795-ac05-36ebe888fb26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.291322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f13b2-d188-4795-ac05-36ebe888fb26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:31.292837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f13b2-d188-4795-ac05-36ebe888fb26 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-c03f13b2-d188-4795-ac05-36ebe888fb26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:31.304225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:31.305140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:31.306163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:31.306163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:31.306163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:31.306163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:31.306163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:31.306163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:31.306163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:31.306163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:31.306595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ad55f-67f1-4950-9a94-381c16b796be None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:31.310560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 514/1036] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:31.311292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f13b2-d188-4795-ac05-36ebe888fb26 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:31.410469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ad55f-67f1-4950-9a94-381c16b796be None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:31.412115 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:31.417284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f13b2-d188-4795-ac05-36ebe888fb26 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:31.446553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bfe5c-2740-418b-8a23-6bff97abdd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:31.447119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bfe5c-2740-418b-8a23-6bff97abdd9a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.447395 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bfe5c-2740-418b-8a23-6bff97abdd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.447661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bfe5c-2740-418b-8a23-6bff97abdd9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.448864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541bfe5c-2740-418b-8a23-6bff97abdd9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:31.450405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541bfe5c-2740-418b-8a23-6bff97abdd9a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:31.451168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:31.452163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 523/1037] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:31.458916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bfe5c-2740-418b-8a23-6bff97abdd9a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:31.461522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d85ec0-9e3e-4e52-871e-423c14ffad1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:31.462220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d85ec0-9e3e-4e52-871e-423c14ffad1b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.462502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d85ec0-9e3e-4e52-871e-423c14ffad1b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.462778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d85ec0-9e3e-4e52-871e-423c14ffad1b None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.463540 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d85ec0-9e3e-4e52-871e-423c14ffad1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:31.463859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d85ec0-9e3e-4e52-871e-423c14ffad1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:31.465448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d85ec0-9e3e-4e52-871e-423c14ffad1b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:31.479084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 515/1038] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:31.485018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d85ec0-9e3e-4e52-871e-423c14ffad1b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:31.564298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bfe5c-2740-418b-8a23-6bff97abdd9a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:31.589303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d85ec0-9e3e-4e52-871e-423c14ffad1b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:31.596878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdc192-21a0-462d-b053-0d874ca21d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:31.597483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdc192-21a0-462d-b053-0d874ca21d3e None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.597754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdc192-21a0-462d-b053-0d874ca21d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.598017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdc192-21a0-462d-b053-0d874ca21d3e None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.598663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbdc192-21a0-462d-b053-0d874ca21d3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:31.600107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbdc192-21a0-462d-b053-0d874ca21d3e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:31.622567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 524/1039] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:24:31 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:24:31.622567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdc192-21a0-462d-b053-0d874ca21d3e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:31.635806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157f5c0-ffd5-479d-aa43-787c8aa4720a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:31.636943 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157f5c0-ffd5-479d-aa43-787c8aa4720a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.637226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157f5c0-ffd5-479d-aa43-787c8aa4720a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.637496 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157f5c0-ffd5-479d-aa43-787c8aa4720a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.638622 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157f5c0-ffd5-479d-aa43-787c8aa4720a None admin] RBAC: Adding query filter params (role.id=814881b0c50648f4a0edcdf280dd29c7, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=2912c02ff6db4cbc97c80cd923238136) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:31.639211 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157f5c0-ffd5-479d-aa43-787c8aa4720a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:31.641013 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157f5c0-ffd5-479d-aa43-787c8aa4720a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:31.659938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 516/1040] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:31 2022] GET /identity/v3/role_assignments?user.id=2912c02ff6db4cbc97c80cd923238136&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=814881b0c50648f4a0edcdf280dd29c7 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:31.662509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:31.746406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdc192-21a0-462d-b053-0d874ca21d3e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:31.794230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078d2e8-cd74-422e-89f6-9eb5e390e39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:31.794895 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078d2e8-cd74-422e-89f6-9eb5e390e39c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.795125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078d2e8-cd74-422e-89f6-9eb5e390e39c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.795387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078d2e8-cd74-422e-89f6-9eb5e390e39c None admin] PATH_INFO: `/v3/services/image` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.796071 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078d2e8-cd74-422e-89f6-9eb5e390e39c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:31.797423 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078d2e8-cd74-422e-89f6-9eb5e390e39c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-0078d2e8-cd74-422e-89f6-9eb5e390e39c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:31.806921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:31.808038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:31.809047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 525/1041] 10.222.0.62 () {56 vars in 1105 bytes} [Thu Sep 1 16:24:31 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:31.814133 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157f5c0-ffd5-479d-aa43-787c8aa4720a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:31.874523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:31.927023 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157f5c0-ffd5-479d-aa43-787c8aa4720a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:31.963010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b5ea5-ccd7-42b1-b2bf-0cad1df64433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:31.963629 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b5ea5-ccd7-42b1-b2bf-0cad1df64433 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:31.963923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b5ea5-ccd7-42b1-b2bf-0cad1df64433 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:31.964194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b5ea5-ccd7-42b1-b2bf-0cad1df64433 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:31.964877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b5ea5-ccd7-42b1-b2bf-0cad1df64433 None admin] RBAC: Adding query filter params (name=image) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:31.965280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b5ea5-ccd7-42b1-b2bf-0cad1df64433 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:31.966770 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b5ea5-ccd7-42b1-b2bf-0cad1df64433 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:31.976692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 517/1042] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:31.978114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078d2e8-cd74-422e-89f6-9eb5e390e39c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:32.074581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:32.085454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078d2e8-cd74-422e-89f6-9eb5e390e39c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:32.120537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659acb36-16d0-475d-858d-7cbede264665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:32.121564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659acb36-16d0-475d-858d-7cbede264665 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:32.122050 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659acb36-16d0-475d-858d-7cbede264665 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:32.122521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659acb36-16d0-475d-858d-7cbede264665 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:32.123334 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659acb36-16d0-475d-858d-7cbede264665 None admin] RBAC: Adding query filter params (type=image) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:32.123830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659acb36-16d0-475d-858d-7cbede264665 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:32.125396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659acb36-16d0-475d-858d-7cbede264665 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:32.131490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 526/1043] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:32.137821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b5ea5-ccd7-42b1-b2bf-0cad1df64433 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:32.247448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b5ea5-ccd7-42b1-b2bf-0cad1df64433 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:32.264717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:32.286774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3747d5d-975e-4e41-ae6a-d04b0b8fdc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:32.287415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3747d5d-975e-4e41-ae6a-d04b0b8fdc1a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:32.287692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3747d5d-975e-4e41-ae6a-d04b0b8fdc1a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:32.287984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3747d5d-975e-4e41-ae6a-d04b0b8fdc1a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:32.288728 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3747d5d-975e-4e41-ae6a-d04b0b8fdc1a None admin] RBAC: Adding query filter params (interface=public, service_id=cb4cf2cdd50444da92f4acdc2652c1cf, region_id=RegionOne) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:32.289035 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3747d5d-975e-4e41-ae6a-d04b0b8fdc1a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:32.290988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3747d5d-975e-4e41-ae6a-d04b0b8fdc1a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:32.300926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 518/1044] 10.222.0.62 () {56 vars in 1251 bytes} [Thu Sep 1 16:24:32 2022] GET /identity/v3/endpoints?service_id=cb4cf2cdd50444da92f4acdc2652c1cf&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:32.302056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659acb36-16d0-475d-858d-7cbede264665 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:32.410707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659acb36-16d0-475d-858d-7cbede264665 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:32.444376 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae27325-8e29-459b-8704-6005b98dca9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:32.445161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae27325-8e29-459b-8704-6005b98dca9d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:32.445580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae27325-8e29-459b-8704-6005b98dca9d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:32.445991 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae27325-8e29-459b-8704-6005b98dca9d None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:32.446767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae27325-8e29-459b-8704-6005b98dca9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:32.448401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae27325-8e29-459b-8704-6005b98dca9d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:32.460352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 527/1045] 10.222.0.62 () {56 vars in 1087 bytes} [Thu Sep 1 16:24:32 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:24:33.540346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac21c09-4558-49d5-848f-2ec37a637533 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:33.541314 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac21c09-4558-49d5-848f-2ec37a637533 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:33.541589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac21c09-4558-49d5-848f-2ec37a637533 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:33.543077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 519/1046] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:33.550078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece684b-daaf-4fef-85c6-97f8bdcbc5d3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:33.550543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece684b-daaf-4fef-85c6-97f8bdcbc5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:33.550620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece684b-daaf-4fef-85c6-97f8bdcbc5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:33.569910 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0ece684b-daaf-4fef-85c6-97f8bdcbc5d3 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:33.571112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0ece684b-daaf-4fef-85c6-97f8bdcbc5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:33.586579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:33.615268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657d9a3-6b4c-41b7-a9d8-4b0fa62859c7 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:33.615606 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657d9a3-6b4c-41b7-a9d8-4b0fa62859c7 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:33.615794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657d9a3-6b4c-41b7-a9d8-4b0fa62859c7 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:33.617410 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 520/1047] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:33.622275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7bc6f-2ef5-4522-b568-bbf017a8ec71 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:33.622551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7bc6f-2ef5-4522-b568-bbf017a8ec71 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:33.622819 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7bc6f-2ef5-4522-b568-bbf017a8ec71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:33.642151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-b5a7bc6f-2ef5-4522-b568-bbf017a8ec71 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:33.643380 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b5a7bc6f-2ef5-4522-b568-bbf017a8ec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:33.647929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 528/1048] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:33.714317 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 521/1049] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:33.913739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece684b-daaf-4fef-85c6-97f8bdcbc5d3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:33.979078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7bc6f-2ef5-4522-b568-bbf017a8ec71 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.030764 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece684b-daaf-4fef-85c6-97f8bdcbc5d3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.076186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aafe9cb-7a8f-4a0c-87bb-ec29919ffea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.076807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafe9cb-7a8f-4a0c-87bb-ec29919ffea1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.077075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafe9cb-7a8f-4a0c-87bb-ec29919ffea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.077534 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafe9cb-7a8f-4a0c-87bb-ec29919ffea1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.080259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7bc6f-2ef5-4522-b568-bbf017a8ec71 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.088306 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aafe9cb-7a8f-4a0c-87bb-ec29919ffea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:34.091780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-3aafe9cb-7a8f-4a0c-87bb-ec29919ffea1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:34.093266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:34.094236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:34.095060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:34.095060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:34.095060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:34.095060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:34.095060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:34.095060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:34.095060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:34.095060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:34.095060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:34.095060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:34.095060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:24:34.095060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:34.095060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 529/1050] 10.222.0.62 () {56 vars in 1120 bytes} [Thu Sep 1 16:24:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:34.100114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aafe9cb-7a8f-4a0c-87bb-ec29919ffea1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.112259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:34.126591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66256f5-90d5-4232-aabc-a11c3715d558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.127445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66256f5-90d5-4232-aabc-a11c3715d558 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.127796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66256f5-90d5-4232-aabc-a11c3715d558 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.128006 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66256f5-90d5-4232-aabc-a11c3715d558 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.139932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66256f5-90d5-4232-aabc-a11c3715d558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-f66256f5-90d5-4232-aabc-a11c3715d558 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:34.145555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:34.146572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:34.147426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:34.147426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:34.147426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:34.147426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:34.147426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:34.147426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:34.147426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:34.147426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:34.147426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:34.147426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:34.147426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:34.147426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:34.152397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 522/1051] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:24:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:34.153266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66256f5-90d5-4232-aabc-a11c3715d558 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.208634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aafe9cb-7a8f-4a0c-87bb-ec29919ffea1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.243427 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46e412-60f6-4ba1-8636-86ac30e7ca05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.244204 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46e412-60f6-4ba1-8636-86ac30e7ca05 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.244490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46e412-60f6-4ba1-8636-86ac30e7ca05 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.244758 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46e412-60f6-4ba1-8636-86ac30e7ca05 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.245673 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46e412-60f6-4ba1-8636-86ac30e7ca05 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:34.245797 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46e412-60f6-4ba1-8636-86ac30e7ca05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:34.247137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46e412-60f6-4ba1-8636-86ac30e7ca05 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:34.255695 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 530/1052] 10.222.0.62 () {56 vars in 1115 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:34.261908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66256f5-90d5-4232-aabc-a11c3715d558 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.262073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46e412-60f6-4ba1-8636-86ac30e7ca05 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.298794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfad8e-3bf0-4773-82f0-94b69d00ea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.299517 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfad8e-3bf0-4773-82f0-94b69d00ea94 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.299797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfad8e-3bf0-4773-82f0-94b69d00ea94 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.300195 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfad8e-3bf0-4773-82f0-94b69d00ea94 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.300878 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adfad8e-3bf0-4773-82f0-94b69d00ea94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:34.301164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adfad8e-3bf0-4773-82f0-94b69d00ea94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:34.302548 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adfad8e-3bf0-4773-82f0-94b69d00ea94 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:34.316891 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 523/1053] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:34.318085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfad8e-3bf0-4773-82f0-94b69d00ea94 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.335073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:34.352913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:34.369755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46e412-60f6-4ba1-8636-86ac30e7ca05 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.402703 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd6662-ad20-4d3a-bda5-bf2dff99c69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.403330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd6662-ad20-4d3a-bda5-bf2dff99c69c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.403605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd6662-ad20-4d3a-bda5-bf2dff99c69c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.403873 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd6662-ad20-4d3a-bda5-bf2dff99c69c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.414098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd6662-ad20-4d3a-bda5-bf2dff99c69c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:34.415638 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd6662-ad20-4d3a-bda5-bf2dff99c69c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:34.421963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfad8e-3bf0-4773-82f0-94b69d00ea94 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2ddd6662-ad20-4d3a-bda5-bf2dff99c69c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:34.429754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:34.430836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:34.432059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:34.432059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:34.432059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:34.432059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 531/1054] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:34.432669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb834451-99a8-4ac9-a8c6-dba20ac8be72 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.433026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb834451-99a8-4ac9-a8c6-dba20ac8be72 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.433302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb834451-99a8-4ac9-a8c6-dba20ac8be72 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.434840 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 532/1055] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:34.438615 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb834451-99a8-4ac9-a8c6-dba20ac8be72 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.460503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff0708-efb9-445d-851a-835706bf08a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.461055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff0708-efb9-445d-851a-835706bf08a1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.461324 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff0708-efb9-445d-851a-835706bf08a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.461578 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff0708-efb9-445d-851a-835706bf08a1 None admin] PATH_INFO: `/v3/users/49397389717f429ab5bc7b087f70c067` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.475045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff0708-efb9-445d-851a-835706bf08a1 None admin] RBAC: Authorizing `identity:get_user(user_id=49397389717f429ab5bc7b087f70c067)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:34.476508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff0708-efb9-445d-851a-835706bf08a1 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:34.478429 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 524/1056] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/users/49397389717f429ab5bc7b087f70c067 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:34.479578 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5fe04-3b6a-4a0a-8f5c-a7923f838cb7 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.479823 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5fe04-3b6a-4a0a-8f5c-a7923f838cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.480127 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5fe04-3b6a-4a0a-8f5c-a7923f838cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.498079 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-51e5fe04-3b6a-4a0a-8f5c-a7923f838cb7 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:34.499247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-51e5fe04-3b6a-4a0a-8f5c-a7923f838cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:34.543707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb834451-99a8-4ac9-a8c6-dba20ac8be72 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.564881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 525/1057] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:34.566368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5fe04-3b6a-4a0a-8f5c-a7923f838cb7 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.582211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:34.585191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:34.598887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76b4bb-0a59-4087-ae3e-3d4834a75195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.599526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76b4bb-0a59-4087-ae3e-3d4834a75195 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.599802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76b4bb-0a59-4087-ae3e-3d4834a75195 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.600111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76b4bb-0a59-4087-ae3e-3d4834a75195 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.600761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76b4bb-0a59-4087-ae3e-3d4834a75195 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:34.601061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76b4bb-0a59-4087-ae3e-3d4834a75195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:34.602522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76b4bb-0a59-4087-ae3e-3d4834a75195 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:34.615419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 533/1058] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:34.621668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76b4bb-0a59-4087-ae3e-3d4834a75195 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.681465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5fe04-3b6a-4a0a-8f5c-a7923f838cb7 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.718674 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69423e67-cb29-48a6-b888-da47b1289b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.719210 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69423e67-cb29-48a6-b888-da47b1289b1e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.719476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69423e67-cb29-48a6-b888-da47b1289b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.719769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69423e67-cb29-48a6-b888-da47b1289b1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.737188 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76b4bb-0a59-4087-ae3e-3d4834a75195 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.774797 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77270251-3492-4d6e-bf08-97904e1f2464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.775435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77270251-3492-4d6e-bf08-97904e1f2464 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.775715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77270251-3492-4d6e-bf08-97904e1f2464 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.776014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77270251-3492-4d6e-bf08-97904e1f2464 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.784043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77270251-3492-4d6e-bf08-97904e1f2464 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:34.785479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77270251-3492-4d6e-bf08-97904e1f2464 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-77270251-3492-4d6e-bf08-97904e1f2464 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:34.796932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:34.798094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:34.799199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:34.799199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:34.799199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:34.799199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:34.799199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:34.799199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:34.799199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:34.799199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:34.799199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:24:34.799199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:34.799199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 534/1059] 10.222.0.62 () {56 vars in 1093 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:34.803680 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77270251-3492-4d6e-bf08-97904e1f2464 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.803798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69423e67-cb29-48a6-b888-da47b1289b1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:34.805225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69423e67-cb29-48a6-b888-da47b1289b1e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:34.825649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:34.849870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:34.905084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77270251-3492-4d6e-bf08-97904e1f2464 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:34.911234 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 526/1060] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:24:34.912369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69423e67-cb29-48a6-b888-da47b1289b1e None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:34.947164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d9ff9-6a11-4cc3-8443-6017035c1931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:34.947765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d9ff9-6a11-4cc3-8443-6017035c1931 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:34.948074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d9ff9-6a11-4cc3-8443-6017035c1931 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:34.948341 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d9ff9-6a11-4cc3-8443-6017035c1931 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:34.949071 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d9ff9-6a11-4cc3-8443-6017035c1931 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:34.949370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d9ff9-6a11-4cc3-8443-6017035c1931 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:34.950800 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d9ff9-6a11-4cc3-8443-6017035c1931 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:34.968603 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 535/1061] 10.222.0.62 () {56 vars in 1133 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:34.969502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d9ff9-6a11-4cc3-8443-6017035c1931 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.015722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69423e67-cb29-48a6-b888-da47b1289b1e None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.032944 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:35.051170 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3f1ab-7467-4211-8be2-5854cac6aa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.051804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3f1ab-7467-4211-8be2-5854cac6aa43 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.052119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3f1ab-7467-4211-8be2-5854cac6aa43 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.052389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3f1ab-7467-4211-8be2-5854cac6aa43 None admin] PATH_INFO: `/v3/services/image` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.052988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3f1ab-7467-4211-8be2-5854cac6aa43 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.054390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3f1ab-7467-4211-8be2-5854cac6aa43 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-5cd3f1ab-7467-4211-8be2-5854cac6aa43 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:35.063531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:35.064748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:35.066134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 527/1062] 10.222.0.62 () {56 vars in 1105 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:35.066962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3f1ab-7467-4211-8be2-5854cac6aa43 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.071080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:35.074804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d9ff9-6a11-4cc3-8443-6017035c1931 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.109982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42133b-d938-4b1f-b96d-b0caea3ce879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.110606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42133b-d938-4b1f-b96d-b0caea3ce879 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.110875 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42133b-d938-4b1f-b96d-b0caea3ce879 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.111138 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42133b-d938-4b1f-b96d-b0caea3ce879 None admin] PATH_INFO: `/v3/projects/ac5674be19c04a06b5d0531f4c50b82f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.112258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42133b-d938-4b1f-b96d-b0caea3ce879 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5674be19c04a06b5d0531f4c50b82f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.113676 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42133b-d938-4b1f-b96d-b0caea3ce879 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.115297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 536/1063] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:24:34 2022] GET /identity/v3/projects/ac5674be19c04a06b5d0531f4c50b82f => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:35.116065 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42133b-d938-4b1f-b96d-b0caea3ce879 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.169392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3f1ab-7467-4211-8be2-5854cac6aa43 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.205800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e79c64-d753-4b91-9560-5c57b2e83232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.206591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e79c64-d753-4b91-9560-5c57b2e83232 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.206881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e79c64-d753-4b91-9560-5c57b2e83232 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.207150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e79c64-d753-4b91-9560-5c57b2e83232 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.219611 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42133b-d938-4b1f-b96d-b0caea3ce879 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.219874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e79c64-d753-4b91-9560-5c57b2e83232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.221483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e79c64-d753-4b91-9560-5c57b2e83232 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-d7e79c64-d753-4b91-9560-5c57b2e83232 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:35.237227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:35.238224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:35.239110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:35.239110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:35.239110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:35.239110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 528/1064] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:35.239604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e79c64-d753-4b91-9560-5c57b2e83232 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.257447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239094f2-43e6-4181-bb6e-bbcbabba178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.258231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239094f2-43e6-4181-bb6e-bbcbabba178c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.258437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239094f2-43e6-4181-bb6e-bbcbabba178c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.258577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239094f2-43e6-4181-bb6e-bbcbabba178c None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.259403 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239094f2-43e6-4181-bb6e-bbcbabba178c None admin] RBAC: Adding query filter params (name=image) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:35.259757 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239094f2-43e6-4181-bb6e-bbcbabba178c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.261253 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239094f2-43e6-4181-bb6e-bbcbabba178c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.266506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 537/1065] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:35.267274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239094f2-43e6-4181-bb6e-bbcbabba178c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.282983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:35.299005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:35.353817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e79c64-d753-4b91-9560-5c57b2e83232 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.368073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239094f2-43e6-4181-bb6e-bbcbabba178c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.388496 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca3d40-20e3-4c18-9834-e6b83d2185d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.389312 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca3d40-20e3-4c18-9834-e6b83d2185d9 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.389583 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca3d40-20e3-4c18-9834-e6b83d2185d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.389852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca3d40-20e3-4c18-9834-e6b83d2185d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.390567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca3d40-20e3-4c18-9834-e6b83d2185d9 None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=ac5674be19c04a06b5d0531f4c50b82f, user.id=49397389717f429ab5bc7b087f70c067) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:35.390866 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca3d40-20e3-4c18-9834-e6b83d2185d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.392342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca3d40-20e3-4c18-9834-e6b83d2185d9 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.398045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 529/1066] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:35 2022] GET /identity/v3/role_assignments?user.id=49397389717f429ab5bc7b087f70c067&scope.project.id=ac5674be19c04a06b5d0531f4c50b82f&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:35.398827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca3d40-20e3-4c18-9834-e6b83d2185d9 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.414379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04f8b6-47bf-4ff6-90cf-88e69850ef3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.415366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04f8b6-47bf-4ff6-90cf-88e69850ef3d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.416152 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04f8b6-47bf-4ff6-90cf-88e69850ef3d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.416152 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04f8b6-47bf-4ff6-90cf-88e69850ef3d None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.416700 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04f8b6-47bf-4ff6-90cf-88e69850ef3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:35.417005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04f8b6-47bf-4ff6-90cf-88e69850ef3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.418638 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04f8b6-47bf-4ff6-90cf-88e69850ef3d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.435687 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 538/1067] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:35.441568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04f8b6-47bf-4ff6-90cf-88e69850ef3d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.521960 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:35.526484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca3d40-20e3-4c18-9834-e6b83d2185d9 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.550784 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:35.569892 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221a8fd-622a-4080-ad85-6a51ab3bc230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.570501 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221a8fd-622a-4080-ad85-6a51ab3bc230 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.570786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221a8fd-622a-4080-ad85-6a51ab3bc230 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.571036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221a8fd-622a-4080-ad85-6a51ab3bc230 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.571745 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221a8fd-622a-4080-ad85-6a51ab3bc230 None admin] RBAC: Adding query filter params (type=image) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:35.572078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221a8fd-622a-4080-ad85-6a51ab3bc230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.573620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04f8b6-47bf-4ff6-90cf-88e69850ef3d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.573970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221a8fd-622a-4080-ad85-6a51ab3bc230 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.579574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 530/1068] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:35.585345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221a8fd-622a-4080-ad85-6a51ab3bc230 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.615397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09dbe2-7c09-4be7-a189-7f87842e8b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.616087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09dbe2-7c09-4be7-a189-7f87842e8b4c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.616320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09dbe2-7c09-4be7-a189-7f87842e8b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.616580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09dbe2-7c09-4be7-a189-7f87842e8b4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.624733 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09dbe2-7c09-4be7-a189-7f87842e8b4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.626199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09dbe2-7c09-4be7-a189-7f87842e8b4c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-7d09dbe2-7c09-4be7-a189-7f87842e8b4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:35.639090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:35.640051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:35.640951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:35.640951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:35.640951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:35.640951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:35.640951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:35.640951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:35.640951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:35.640951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:35.640951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 539/1069] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:35.645886 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09dbe2-7c09-4be7-a189-7f87842e8b4c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.695411 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221a8fd-622a-4080-ad85-6a51ab3bc230 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.732269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e4207-6d00-401d-9c4c-b089388b144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.732870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e4207-6d00-401d-9c4c-b089388b144c None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.733150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e4207-6d00-401d-9c4c-b089388b144c None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.733414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e4207-6d00-401d-9c4c-b089388b144c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.734120 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e4207-6d00-401d-9c4c-b089388b144c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.735477 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e4207-6d00-401d-9c4c-b089388b144c None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.767927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 531/1070] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:24:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:24:35.768129 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09dbe2-7c09-4be7-a189-7f87842e8b4c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.814734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1d3b4-8592-49c7-8611-e46a78791ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.815352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1d3b4-8592-49c7-8611-e46a78791ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.815622 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1d3b4-8592-49c7-8611-e46a78791ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.815907 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1d3b4-8592-49c7-8611-e46a78791ec6 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.816615 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1d3b4-8592-49c7-8611-e46a78791ec6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:35.816916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1d3b4-8592-49c7-8611-e46a78791ec6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.818394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1d3b4-8592-49c7-8611-e46a78791ec6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:35.838385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 540/1071] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:35.839915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e4207-6d00-401d-9c4c-b089388b144c None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:35.864186 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:35.958804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e4207-6d00-401d-9c4c-b089388b144c None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:35.991233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374caf2-6916-462d-a693-926167f6f7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:35.991950 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374caf2-6916-462d-a693-926167f6f7c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.992231 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374caf2-6916-462d-a693-926167f6f7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.992503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374caf2-6916-462d-a693-926167f6f7c4 None admin] PATH_INFO: `/v3/projects/425048ffd13448cd8fe815333a9b798c/users/2912c02ff6db4cbc97c80cd923238136/roles/814881b0c50648f4a0edcdf280dd29c7` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.994972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374caf2-6916-462d-a693-926167f6f7c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=425048ffd13448cd8fe815333a9b798c, user_id=2912c02ff6db4cbc97c80cd923238136, role_id=814881b0c50648f4a0edcdf280dd29c7)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:35.996470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374caf2-6916-462d-a693-926167f6f7c4 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:36.011143 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 532/1072] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:24:35 2022] PUT /identity/v3/projects/425048ffd13448cd8fe815333a9b798c/users/2912c02ff6db4cbc97c80cd923238136/roles/814881b0c50648f4a0edcdf280dd29c7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:24:37.318527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e74d5a-85e4-4678-8793-39450cded41a None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:37.319153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e74d5a-85e4-4678-8793-39450cded41a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:37.319153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e74d5a-85e4-4678-8793-39450cded41a None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:37.320494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 541/1073] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:37.326691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe0cff-64dd-48fb-920f-bd273ca0ab53 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:37.327167 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe0cff-64dd-48fb-920f-bd273ca0ab53 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:37.327245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe0cff-64dd-48fb-920f-bd273ca0ab53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:37.351402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-1efe0cff-64dd-48fb-920f-bd273ca0ab53 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:37.353187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1efe0cff-64dd-48fb-920f-bd273ca0ab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:37.427032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 533/1074] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:37.691171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e74d5a-85e4-4678-8793-39450cded41a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:37.703519 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:37.746387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377d5be-8801-49c2-a3a9-b5f07b30c761 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:37.746739 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377d5be-8801-49c2-a3a9-b5f07b30c761 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:37.746931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377d5be-8801-49c2-a3a9-b5f07b30c761 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:37.749187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 534/1075] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:37.753877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25277ccf-6ff2-48cb-98e3-bd5cc55cec8a None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:37.754162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25277ccf-6ff2-48cb-98e3-bd5cc55cec8a None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:37.754433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25277ccf-6ff2-48cb-98e3-bd5cc55cec8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:37.773819 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-25277ccf-6ff2-48cb-98e3-bd5cc55cec8a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:37.775155 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-25277ccf-6ff2-48cb-98e3-bd5cc55cec8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:37.777369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:37.809909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e74d5a-85e4-4678-8793-39450cded41a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:37.849394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fab127-ed7f-4698-b71c-5c989f1cbb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:37.850028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab127-ed7f-4698-b71c-5c989f1cbb2e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:37.850301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab127-ed7f-4698-b71c-5c989f1cbb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:37.850563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab127-ed7f-4698-b71c-5c989f1cbb2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:37.852341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 535/1076] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:37.861494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fab127-ed7f-4698-b71c-5c989f1cbb2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-39fab127-ed7f-4698-b71c-5c989f1cbb2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:37.866299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:37.867207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:37.868199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:37.868199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:37.868199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:37.868199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:37.868199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:37.868199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:37.868199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:37.868199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:37.868199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:37.868199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:37.868199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:37.868199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:37.868199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 542/1077] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:24:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:37.874109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25277ccf-6ff2-48cb-98e3-bd5cc55cec8a None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:37.901134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37332ba7-cef1-414d-90e5-d452b41c8036 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:37.901418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37332ba7-cef1-414d-90e5-d452b41c8036 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:37.901689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37332ba7-cef1-414d-90e5-d452b41c8036 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:37.903714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 543/1078] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:24:37.908362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66929558-224e-44ad-9a99-80714fbdd28f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:37.908652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66929558-224e-44ad-9a99-80714fbdd28f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:37.908896 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66929558-224e-44ad-9a99-80714fbdd28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:37.928264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-66929558-224e-44ad-9a99-80714fbdd28f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:37.929559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-66929558-224e-44ad-9a99-80714fbdd28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:37.978317 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25277ccf-6ff2-48cb-98e3-bd5cc55cec8a None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:37.995580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:38.002451 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 544/1079] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:38.017746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158924d6-ca81-47f0-852d-cd25f8df5e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.018658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158924d6-ca81-47f0-852d-cd25f8df5e67 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.019129 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158924d6-ca81-47f0-852d-cd25f8df5e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.019539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158924d6-ca81-47f0-852d-cd25f8df5e67 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.020458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158924d6-ca81-47f0-852d-cd25f8df5e67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:38.020902 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158924d6-ca81-47f0-852d-cd25f8df5e67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.022494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158924d6-ca81-47f0-852d-cd25f8df5e67 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.030126 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:38.034401 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 536/1080] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:24:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:38.039948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66929558-224e-44ad-9a99-80714fbdd28f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.122547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158924d6-ca81-47f0-852d-cd25f8df5e67 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.141665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66929558-224e-44ad-9a99-80714fbdd28f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.177997 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c457a-4843-47b1-a1df-4f041dff9ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.178696 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c457a-4843-47b1-a1df-4f041dff9ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.178896 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c457a-4843-47b1-a1df-4f041dff9ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.179170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c457a-4843-47b1-a1df-4f041dff9ea3 None admin] PATH_INFO: `/v3/users/49397389717f429ab5bc7b087f70c067` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.180374 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c457a-4843-47b1-a1df-4f041dff9ea3 None admin] RBAC: Authorizing `identity:get_user(user_id=49397389717f429ab5bc7b087f70c067)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.181850 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c457a-4843-47b1-a1df-4f041dff9ea3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.188035 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 545/1081] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:24:38 2022] GET /identity/v3/users/49397389717f429ab5bc7b087f70c067 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:38.189656 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c457a-4843-47b1-a1df-4f041dff9ea3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.233934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158924d6-ca81-47f0-852d-cd25f8df5e67 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.261070 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:38.273151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7acc5-c5dc-48d3-8d2c-aef623caf68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.273767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7acc5-c5dc-48d3-8d2c-aef623caf68a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.274040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7acc5-c5dc-48d3-8d2c-aef623caf68a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.274471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7acc5-c5dc-48d3-8d2c-aef623caf68a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.299318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c457a-4843-47b1-a1df-4f041dff9ea3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.332475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83383f8b-962e-4520-a9a5-42a6ae6f6de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.333420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83383f8b-962e-4520-a9a5-42a6ae6f6de6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.333690 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83383f8b-962e-4520-a9a5-42a6ae6f6de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.333958 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83383f8b-962e-4520-a9a5-42a6ae6f6de6 None admin] PATH_INFO: `/v3/projects/ac5674be19c04a06b5d0531f4c50b82f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.335074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83383f8b-962e-4520-a9a5-42a6ae6f6de6 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5674be19c04a06b5d0531f4c50b82f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.336554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83383f8b-962e-4520-a9a5-42a6ae6f6de6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.338334 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 546/1082] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:24:38 2022] GET /identity/v3/projects/ac5674be19c04a06b5d0531f4c50b82f => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:38.339118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83383f8b-962e-4520-a9a5-42a6ae6f6de6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.344653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:38.347349 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7acc5-c5dc-48d3-8d2c-aef623caf68a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.348948 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7acc5-c5dc-48d3-8d2c-aef623caf68a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.437445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83383f8b-962e-4520-a9a5-42a6ae6f6de6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.450167 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 537/1083] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:24:38.451014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7acc5-c5dc-48d3-8d2c-aef623caf68a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.473673 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd02812-18c8-4cac-a65c-fa608e9385ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.474307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd02812-18c8-4cac-a65c-fa608e9385ec None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.474585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd02812-18c8-4cac-a65c-fa608e9385ec None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.474865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd02812-18c8-4cac-a65c-fa608e9385ec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.476732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:38.486257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd02812-18c8-4cac-a65c-fa608e9385ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-bdd02812-18c8-4cac-a65c-fa608e9385ec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:38.491501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:38.492461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:38.493407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:38.493407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:38.493407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:38.493407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:38.493407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:38.493407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:38.493407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:38.493407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:38.493407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:38.493407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:38.493407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:24:38.493407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:38.493407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 547/1084] 10.222.0.62 () {56 vars in 1120 bytes} [Thu Sep 1 16:24:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:38.494108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd02812-18c8-4cac-a65c-fa608e9385ec None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.550894 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:38.554012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7acc5-c5dc-48d3-8d2c-aef623caf68a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.589674 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17892d-9bc8-4008-be4e-aa6687648d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.590429 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17892d-9bc8-4008-be4e-aa6687648d43 None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.590776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17892d-9bc8-4008-be4e-aa6687648d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.591043 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17892d-9bc8-4008-be4e-aa6687648d43 None admin] PATH_INFO: `/v3/projects/ac5674be19c04a06b5d0531f4c50b82f/users/49397389717f429ab5bc7b087f70c067/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.593047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd02812-18c8-4cac-a65c-fa608e9385ec None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.593162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17892d-9bc8-4008-be4e-aa6687648d43 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac5674be19c04a06b5d0531f4c50b82f, user_id=49397389717f429ab5bc7b087f70c067, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.594490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17892d-9bc8-4008-be4e-aa6687648d43 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.608097 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 538/1085] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:24:38 2022] PUT /identity/v3/projects/ac5674be19c04a06b5d0531f4c50b82f/users/49397389717f429ab5bc7b087f70c067/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:24:38.608862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17892d-9bc8-4008-be4e-aa6687648d43 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.643533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77977c3-da2f-4745-b34f-cf8d5d93fd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.644611 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77977c3-da2f-4745-b34f-cf8d5d93fd67 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.644892 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77977c3-da2f-4745-b34f-cf8d5d93fd67 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.645155 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77977c3-da2f-4745-b34f-cf8d5d93fd67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.659302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77977c3-da2f-4745-b34f-cf8d5d93fd67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.660829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77977c3-da2f-4745-b34f-cf8d5d93fd67 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-f77977c3-da2f-4745-b34f-cf8d5d93fd67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:38.677580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:38.678682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:38.679704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:38.679704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:38.679704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:38.679704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 548/1086] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:38.685210 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77977c3-da2f-4745-b34f-cf8d5d93fd67 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.733664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17892d-9bc8-4008-be4e-aa6687648d43 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.747719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:38.784613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f6cd4-2bf9-487c-a8d2-d8a8eaf2f268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.785228 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f6cd4-2bf9-487c-a8d2-d8a8eaf2f268 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.785501 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f6cd4-2bf9-487c-a8d2-d8a8eaf2f268 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.785762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f6cd4-2bf9-487c-a8d2-d8a8eaf2f268 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.786710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f6cd4-2bf9-487c-a8d2-d8a8eaf2f268 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:38.786710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f6cd4-2bf9-487c-a8d2-d8a8eaf2f268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.788099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f6cd4-2bf9-487c-a8d2-d8a8eaf2f268 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.797300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 539/1087] 10.222.0.62 () {56 vars in 1115 bytes} [Thu Sep 1 16:24:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:38.803237 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f6cd4-2bf9-487c-a8d2-d8a8eaf2f268 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:38.810403 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77977c3-da2f-4745-b34f-cf8d5d93fd67 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.834798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:38.844889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7218237-e82a-409c-9153-a4cb1d35f5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.845559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7218237-e82a-409c-9153-a4cb1d35f5da None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.845841 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7218237-e82a-409c-9153-a4cb1d35f5da None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.846113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7218237-e82a-409c-9153-a4cb1d35f5da None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.846785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7218237-e82a-409c-9153-a4cb1d35f5da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:38.847081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7218237-e82a-409c-9153-a4cb1d35f5da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.848558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7218237-e82a-409c-9153-a4cb1d35f5da None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.861182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 549/1088] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:38.929553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f6cd4-2bf9-487c-a8d2-d8a8eaf2f268 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:38.968882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77bdb4-b06c-4fa7-9524-83fd6e4ffc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:38.969540 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77bdb4-b06c-4fa7-9524-83fd6e4ffc67 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:38.969795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77bdb4-b06c-4fa7-9524-83fd6e4ffc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:38.970101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77bdb4-b06c-4fa7-9524-83fd6e4ffc67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:38.982713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77bdb4-b06c-4fa7-9524-83fd6e4ffc67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:38.984223 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77bdb4-b06c-4fa7-9524-83fd6e4ffc67 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-6d77bdb4-b06c-4fa7-9524-83fd6e4ffc67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:38.999979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:39.000992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:39.001885 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:39.001885 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:39.001885 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:39.003260 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 540/1089] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:24:39.007362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7218237-e82a-409c-9153-a4cb1d35f5da None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:39.124054 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7218237-e82a-409c-9153-a4cb1d35f5da None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:39.131857 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:39.160747 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ae639-1286-4210-9258-f1cf9c0e3ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:39.161351 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ae639-1286-4210-9258-f1cf9c0e3ebc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.161624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ae639-1286-4210-9258-f1cf9c0e3ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.161889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ae639-1286-4210-9258-f1cf9c0e3ebc None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.162519 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ae639-1286-4210-9258-f1cf9c0e3ebc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:39.162817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ae639-1286-4210-9258-f1cf9c0e3ebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:39.164210 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ae639-1286-4210-9258-f1cf9c0e3ebc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:39.176134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 550/1090] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:24:39.181717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77bdb4-b06c-4fa7-9524-83fd6e4ffc67 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:39.255031 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:39.282140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77bdb4-b06c-4fa7-9524-83fd6e4ffc67 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:39.316857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648d682-9019-4647-a49b-c3ffbffafd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:39.317486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648d682-9019-4647-a49b-c3ffbffafd73 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.317764 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648d682-9019-4647-a49b-c3ffbffafd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.318031 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648d682-9019-4647-a49b-c3ffbffafd73 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.325672 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648d682-9019-4647-a49b-c3ffbffafd73 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:39.327125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648d682-9019-4647-a49b-c3ffbffafd73 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-5648d682-9019-4647-a49b-c3ffbffafd73 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:24:39.338625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:39.339561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:39.340544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:39.340544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:39.340544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:24:39.340544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:24:39.340544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:24:39.340544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:24:39.340544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:24:39.340544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:24:39.340544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:24:39.340544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:39.344056 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 541/1091] 10.222.0.62 () {56 vars in 1093 bytes} [Thu Sep 1 16:24:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:39.345557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ae639-1286-4210-9258-f1cf9c0e3ebc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:39.447937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ae639-1286-4210-9258-f1cf9c0e3ebc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:39.483340 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c02d0-9879-468a-b713-0ca773fe0ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:39.483943 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c02d0-9879-468a-b713-0ca773fe0ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.484252 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c02d0-9879-468a-b713-0ca773fe0ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.484516 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c02d0-9879-468a-b713-0ca773fe0ad5 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.485244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c02d0-9879-468a-b713-0ca773fe0ad5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:39.485545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c02d0-9879-468a-b713-0ca773fe0ad5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:39.486977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c02d0-9879-468a-b713-0ca773fe0ad5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:39.505801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 551/1092] 10.222.0.62 () {56 vars in 1133 bytes} [Thu Sep 1 16:24:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:39.511921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648d682-9019-4647-a49b-c3ffbffafd73 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:39.613659 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648d682-9019-4647-a49b-c3ffbffafd73 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:39.649664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff525d1-420d-4ebe-b637-dff9f08454fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:39.650350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff525d1-420d-4ebe-b637-dff9f08454fe None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.650623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff525d1-420d-4ebe-b637-dff9f08454fe None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.650886 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff525d1-420d-4ebe-b637-dff9f08454fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.653550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:39.723925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff525d1-420d-4ebe-b637-dff9f08454fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:39.725414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff525d1-420d-4ebe-b637-dff9f08454fe None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:39.825142 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 542/1093] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:24:39.831372 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c02d0-9879-468a-b713-0ca773fe0ad5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:39.857788 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:39.937268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c02d0-9879-468a-b713-0ca773fe0ad5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:39.973992 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d2cd40-2a1b-4d5e-a50f-1c4c7037a115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:39.974624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2cd40-2a1b-4d5e-a50f-1c4c7037a115 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.974904 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2cd40-2a1b-4d5e-a50f-1c4c7037a115 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.975177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2cd40-2a1b-4d5e-a50f-1c4c7037a115 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.986223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d2cd40-2a1b-4d5e-a50f-1c4c7037a115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:39.987717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d2cd40-2a1b-4d5e-a50f-1c4c7037a115 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b2d2cd40-2a1b-4d5e-a50f-1c4c7037a115 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:24:40.002443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:40.003396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:24:40.004348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:24:40.004348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:24:40.004348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:40.004348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 552/1094] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:40.009460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff525d1-420d-4ebe-b637-dff9f08454fe None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:40.044435 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:40.113507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff525d1-420d-4ebe-b637-dff9f08454fe None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:40.147347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2075b8-16c6-4d77-9ac8-2162a6c22f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:40.147942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2075b8-16c6-4d77-9ac8-2162a6c22f0b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:40.148244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2075b8-16c6-4d77-9ac8-2162a6c22f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:40.148507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2075b8-16c6-4d77-9ac8-2162a6c22f0b None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:40.149139 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2075b8-16c6-4d77-9ac8-2162a6c22f0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:40.149480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2075b8-16c6-4d77-9ac8-2162a6c22f0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:40.150848 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2075b8-16c6-4d77-9ac8-2162a6c22f0b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:40.162276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 543/1095] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:40.168068 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d2cd40-2a1b-4d5e-a50f-1c4c7037a115 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:40.246645 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:40.269736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d2cd40-2a1b-4d5e-a50f-1c4c7037a115 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:40.303434 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87188f33-6e37-4228-9290-e019e3d92a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:40.304011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87188f33-6e37-4228-9290-e019e3d92a3a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:40.304284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87188f33-6e37-4228-9290-e019e3d92a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:40.304548 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87188f33-6e37-4228-9290-e019e3d92a3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:40.311200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87188f33-6e37-4228-9290-e019e3d92a3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:40.312703 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87188f33-6e37-4228-9290-e019e3d92a3a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-87188f33-6e37-4228-9290-e019e3d92a3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:24:40.323669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:40.324606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:40.325501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:24:40.325501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:24:40.325501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:24:40.325501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:24:40.325501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:24:40.325501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:24:40.325501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:24:40.325501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:24:40.325501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 553/1096] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:24:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:40.330691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2075b8-16c6-4d77-9ac8-2162a6c22f0b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:40.425298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:40.432908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2075b8-16c6-4d77-9ac8-2162a6c22f0b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:40.469020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415b081-3a2d-46d7-84db-f533f5ada5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:40.469624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415b081-3a2d-46d7-84db-f533f5ada5ac None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:40.469897 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415b081-3a2d-46d7-84db-f533f5ada5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:40.470162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415b081-3a2d-46d7-84db-f533f5ada5ac None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:40.470844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415b081-3a2d-46d7-84db-f533f5ada5ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:40.471139 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415b081-3a2d-46d7-84db-f533f5ada5ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:40.472605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415b081-3a2d-46d7-84db-f533f5ada5ac None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:40.484842 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 544/1097] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:24:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:40.491011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87188f33-6e37-4228-9290-e019e3d92a3a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:40.522166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db9799-20b3-4a23-863d-76f6f815d40b None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:40.522705 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db9799-20b3-4a23-863d-76f6f815d40b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:40.522789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db9799-20b3-4a23-863d-76f6f815d40b None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:40.524558 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 545/1098] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:40.530575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503576f7-5e14-48f9-a8e0-69f7b651e10a None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:40.530853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503576f7-5e14-48f9-a8e0-69f7b651e10a None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:40.531120 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503576f7-5e14-48f9-a8e0-69f7b651e10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:40.551200 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-503576f7-5e14-48f9-a8e0-69f7b651e10a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:40.552446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-503576f7-5e14-48f9-a8e0-69f7b651e10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:40.594942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87188f33-6e37-4228-9290-e019e3d92a3a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:40.621948 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 546/1099] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:40.629394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb46ba-6bca-47b7-bad7-b104e7e8a781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:40.630000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb46ba-6bca-47b7-bad7-b104e7e8a781 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:40.630277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb46ba-6bca-47b7-bad7-b104e7e8a781 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:40.630543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb46ba-6bca-47b7-bad7-b104e7e8a781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:40.631263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb46ba-6bca-47b7-bad7-b104e7e8a781 None admin] RBAC: Adding query filter params (role.id=814881b0c50648f4a0edcdf280dd29c7, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=2912c02ff6db4cbc97c80cd923238136) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:40.631571 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb46ba-6bca-47b7-bad7-b104e7e8a781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:40.633026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb46ba-6bca-47b7-bad7-b104e7e8a781 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:40.637981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:40.641382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 554/1100] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:40 2022] GET /identity/v3/role_assignments?user.id=2912c02ff6db4cbc97c80cd923238136&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=814881b0c50648f4a0edcdf280dd29c7 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:40.886598 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503576f7-5e14-48f9-a8e0-69f7b651e10a None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:40.976836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:40.996601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503576f7-5e14-48f9-a8e0-69f7b651e10a None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:41.032181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1b617-1d74-467c-9545-038eb421ba47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:41.033109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1b617-1d74-467c-9545-038eb421ba47 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:41.033521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1b617-1d74-467c-9545-038eb421ba47 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:41.033956 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1b617-1d74-467c-9545-038eb421ba47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:41.045497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1b617-1d74-467c-9545-038eb421ba47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-c5c1b617-1d74-467c-9545-038eb421ba47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:24:41.050382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:24:41.051331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:24:41.052239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:24:41.052239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:24:41.052239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:24:41.052239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:24:41.052239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:24:41.052239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:24:41.052239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:24:41.052239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:24:41.052239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:24:41.052239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:24:41.052239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:24:41.052239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:24:41.052239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 547/1101] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:24:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:24:41.058495 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb46ba-6bca-47b7-bad7-b104e7e8a781 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:41.167405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb46ba-6bca-47b7-bad7-b104e7e8a781 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:41.202050 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02dd19-b3ad-4621-bbd7-6ff4c0800817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:41.202668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02dd19-b3ad-4621-bbd7-6ff4c0800817 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:41.202947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02dd19-b3ad-4621-bbd7-6ff4c0800817 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:41.203214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02dd19-b3ad-4621-bbd7-6ff4c0800817 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:41.203875 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02dd19-b3ad-4621-bbd7-6ff4c0800817 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:41.204206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02dd19-b3ad-4621-bbd7-6ff4c0800817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:41.205596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02dd19-b3ad-4621-bbd7-6ff4c0800817 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:41.214880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 555/1102] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:24:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:41.220893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1b617-1d74-467c-9545-038eb421ba47 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:41.326710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1b617-1d74-467c-9545-038eb421ba47 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:41.364109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1a4fc-8b79-4ad4-9a03-dca94d643f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:41.364749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1a4fc-8b79-4ad4-9a03-dca94d643f10 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:41.365023 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1a4fc-8b79-4ad4-9a03-dca94d643f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:41.365288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1a4fc-8b79-4ad4-9a03-dca94d643f10 None admin] PATH_INFO: `/v3/users/49397389717f429ab5bc7b087f70c067` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:41.366422 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1a4fc-8b79-4ad4-9a03-dca94d643f10 None admin] RBAC: Authorizing `identity:get_user(user_id=49397389717f429ab5bc7b087f70c067)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:41.367918 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1a4fc-8b79-4ad4-9a03-dca94d643f10 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:41.369545 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 548/1103] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:24:41 2022] GET /identity/v3/users/49397389717f429ab5bc7b087f70c067 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:41.376328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02dd19-b3ad-4621-bbd7-6ff4c0800817 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:41.453889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:41.493532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02dd19-b3ad-4621-bbd7-6ff4c0800817 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:41.527041 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a511d6-a9f5-4c27-8d92-f991b3033d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:41.527637 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a511d6-a9f5-4c27-8d92-f991b3033d73 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:41.527912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a511d6-a9f5-4c27-8d92-f991b3033d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:41.528279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a511d6-a9f5-4c27-8d92-f991b3033d73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:41.596900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a511d6-a9f5-4c27-8d92-f991b3033d73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:41.598325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a511d6-a9f5-4c27-8d92-f991b3033d73 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:41.667303 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:41.710572 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 556/1104] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:24:41.716962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1a4fc-8b79-4ad4-9a03-dca94d643f10 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:41.734461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:41.841929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1a4fc-8b79-4ad4-9a03-dca94d643f10 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:41.878047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b30a5-1ba6-408c-83d6-56301aa3de76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:41.878659 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b30a5-1ba6-408c-83d6-56301aa3de76 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:41.878934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b30a5-1ba6-408c-83d6-56301aa3de76 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:41.879199 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b30a5-1ba6-408c-83d6-56301aa3de76 None admin] PATH_INFO: `/v3/projects/ac5674be19c04a06b5d0531f4c50b82f` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:41.880459 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b30a5-1ba6-408c-83d6-56301aa3de76 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5674be19c04a06b5d0531f4c50b82f)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:41.881890 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b30a5-1ba6-408c-83d6-56301aa3de76 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:41.883569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 549/1105] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:24:41 2022] GET /identity/v3/projects/ac5674be19c04a06b5d0531f4c50b82f => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:41.890913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a511d6-a9f5-4c27-8d92-f991b3033d73 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:42.003958 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a511d6-a9f5-4c27-8d92-f991b3033d73 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:42.041549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0849c2-c95b-42ce-9a97-00809abc56fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:42.042146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0849c2-c95b-42ce-9a97-00809abc56fd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:42.042419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0849c2-c95b-42ce-9a97-00809abc56fd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:42.042687 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0849c2-c95b-42ce-9a97-00809abc56fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:42.043402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0849c2-c95b-42ce-9a97-00809abc56fd None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=ac5674be19c04a06b5d0531f4c50b82f, user.id=49397389717f429ab5bc7b087f70c067) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:42.043701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0849c2-c95b-42ce-9a97-00809abc56fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:42.045150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0849c2-c95b-42ce-9a97-00809abc56fd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:42.050944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 557/1106] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:41 2022] GET /identity/v3/role_assignments?user.id=49397389717f429ab5bc7b087f70c067&scope.project.id=ac5674be19c04a06b5d0531f4c50b82f&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:43.959170 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fbad3-7f3f-45ea-96bd-7ea8db259d11 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:43.959737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fbad3-7f3f-45ea-96bd-7ea8db259d11 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:43.959737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fbad3-7f3f-45ea-96bd-7ea8db259d11 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:43.961125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 550/1107] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:43.967652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc23af-079c-4387-8a4c-c6d1bcee1610 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:43.968444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc23af-079c-4387-8a4c-c6d1bcee1610 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:43.968580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc23af-079c-4387-8a4c-c6d1bcee1610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:43.997014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-63fc23af-079c-4387-8a4c-c6d1bcee1610 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:43.998190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-63fc23af-079c-4387-8a4c-c6d1bcee1610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:44.019608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:44.069622 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 558/1108] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:44.347746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8fbad3-7f3f-45ea-96bd-7ea8db259d11 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:44.450655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:44.472416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8fbad3-7f3f-45ea-96bd-7ea8db259d11 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:44.508809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fd21f-2642-4806-b67b-923e2eee1bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:44.509565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fd21f-2642-4806-b67b-923e2eee1bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:44.509839 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fd21f-2642-4806-b67b-923e2eee1bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:44.510100 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fd21f-2642-4806-b67b-923e2eee1bd9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:44.511328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8fd21f-2642-4806-b67b-923e2eee1bd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:44.512791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8fd21f-2642-4806-b67b-923e2eee1bd9 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:44.514355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 551/1109] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:44.520448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc23af-079c-4387-8a4c-c6d1bcee1610 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:44.623202 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc23af-079c-4387-8a4c-c6d1bcee1610 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:44.659275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9b715-a711-4de1-9731-aa3ef70ace66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:44.659872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9b715-a711-4de1-9731-aa3ef70ace66 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:44.660164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9b715-a711-4de1-9731-aa3ef70ace66 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:44.660411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9b715-a711-4de1-9731-aa3ef70ace66 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:44.661044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9b715-a711-4de1-9731-aa3ef70ace66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:44.662496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9b715-a711-4de1-9731-aa3ef70ace66 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:44.686792 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 559/1110] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:24:44 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:24:46.627313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb23fac-463e-46de-b9d3-62d070692353 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:46.628529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb23fac-463e-46de-b9d3-62d070692353 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:46.628529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb23fac-463e-46de-b9d3-62d070692353 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:46.630415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 552/1111] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:46.636667 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7d960-2526-4386-ad05-9c5190b1bb1d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:46.637131 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7d960-2526-4386-ad05-9c5190b1bb1d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:46.637237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7d960-2526-4386-ad05-9c5190b1bb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:46.657983 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bab7d960-2526-4386-ad05-9c5190b1bb1d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:46.659215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bab7d960-2526-4386-ad05-9c5190b1bb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:46.667119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:46.735685 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 560/1112] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:47.004283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb23fac-463e-46de-b9d3-62d070692353 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:47.120322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb23fac-463e-46de-b9d3-62d070692353 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:47.162491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b00f0-28ad-44df-98dc-27eb51ba48b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:47.163113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b00f0-28ad-44df-98dc-27eb51ba48b8 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:47.163390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b00f0-28ad-44df-98dc-27eb51ba48b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:47.163651 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b00f0-28ad-44df-98dc-27eb51ba48b8 None admin] PATH_INFO: `/v3/roles/be3ffa4db9954414868f116f97f3a929` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:47.165160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b00f0-28ad-44df-98dc-27eb51ba48b8 None admin] RBAC: Authorizing `identity:get_role(role_id=be3ffa4db9954414868f116f97f3a929)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:47.166577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b00f0-28ad-44df-98dc-27eb51ba48b8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:47.167942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 553/1113] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:24:47 2022] GET /identity/v3/roles/be3ffa4db9954414868f116f97f3a929 => generated 223 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:47.175586 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7d960-2526-4386-ad05-9c5190b1bb1d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:47.300941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7d960-2526-4386-ad05-9c5190b1bb1d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:47.341124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fa4c3-1244-464f-ad64-75e430a0a517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:47.342080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fa4c3-1244-464f-ad64-75e430a0a517 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:47.342499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fa4c3-1244-464f-ad64-75e430a0a517 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:47.342960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fa4c3-1244-464f-ad64-75e430a0a517 None admin] PATH_INFO: `/v3/users/0bd45467d9c746cfaa40b7b0f7cdf44b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:47.355963 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:47.358878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23fa4c3-1244-464f-ad64-75e430a0a517 None admin] RBAC: Authorizing `identity:get_user(user_id=0bd45467d9c746cfaa40b7b0f7cdf44b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:47.360690 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23fa4c3-1244-464f-ad64-75e430a0a517 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:47.362317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 561/1114] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:24:47 2022] GET /identity/v3/users/0bd45467d9c746cfaa40b7b0f7cdf44b => generated 286 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:47.368686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b00f0-28ad-44df-98dc-27eb51ba48b8 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:47.389562 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:47.481805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b00f0-28ad-44df-98dc-27eb51ba48b8 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:47.522573 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e0b4a-fd91-410d-89a2-a76803688595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:47.523388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e0b4a-fd91-410d-89a2-a76803688595 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:47.523791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e0b4a-fd91-410d-89a2-a76803688595 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:47.524187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e0b4a-fd91-410d-89a2-a76803688595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:47.596785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e0b4a-fd91-410d-89a2-a76803688595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:47.598241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e0b4a-fd91-410d-89a2-a76803688595 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:47.615248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:47.706708 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 554/1115] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:24:47.714928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fa4c3-1244-464f-ad64-75e430a0a517 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:47.840269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fa4c3-1244-464f-ad64-75e430a0a517 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:47.876288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87d9aa-b3a9-4227-a0d0-cee1b60a56a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:47.876878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87d9aa-b3a9-4227-a0d0-cee1b60a56a7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:47.877156 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87d9aa-b3a9-4227-a0d0-cee1b60a56a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:47.877416 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87d9aa-b3a9-4227-a0d0-cee1b60a56a7 None admin] PATH_INFO: `/v3/projects/ac5674be19c04a06b5d0531f4c50b82f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:47.878641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87d9aa-b3a9-4227-a0d0-cee1b60a56a7 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5674be19c04a06b5d0531f4c50b82f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:47.880175 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87d9aa-b3a9-4227-a0d0-cee1b60a56a7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:47.881872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 562/1116] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:24:47 2022] GET /identity/v3/projects/ac5674be19c04a06b5d0531f4c50b82f => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:47.889204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e0b4a-fd91-410d-89a2-a76803688595 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:48.002029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e0b4a-fd91-410d-89a2-a76803688595 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:48.018720 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:48.038337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5b781-efa4-49aa-95ee-1173435ee87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:48.038950 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5b781-efa4-49aa-95ee-1173435ee87e None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:48.039209 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5b781-efa4-49aa-95ee-1173435ee87e None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:48.039475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5b781-efa4-49aa-95ee-1173435ee87e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:48.040314 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5b781-efa4-49aa-95ee-1173435ee87e None admin] RBAC: Adding query filter params (role.id=be3ffa4db9954414868f116f97f3a929, scope.project.id=ac5674be19c04a06b5d0531f4c50b82f, user.id=0bd45467d9c746cfaa40b7b0f7cdf44b) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:48.040620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5b781-efa4-49aa-95ee-1173435ee87e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:48.042062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5b781-efa4-49aa-95ee-1173435ee87e None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:48.047425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 555/1117] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:47 2022] GET /identity/v3/role_assignments?user.id=0bd45467d9c746cfaa40b7b0f7cdf44b&scope.project.id=ac5674be19c04a06b5d0531f4c50b82f&role.id=be3ffa4db9954414868f116f97f3a929 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:49.928439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bb186-4be1-4bdb-a928-8f8843a73711 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:49.929308 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bb186-4be1-4bdb-a928-8f8843a73711 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:49.929308 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bb186-4be1-4bdb-a928-8f8843a73711 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:49.930675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 563/1118] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:49.937965 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39852091-eccc-432f-94a6-11d9494089b5 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:49.938466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39852091-eccc-432f-94a6-11d9494089b5 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:49.938466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39852091-eccc-432f-94a6-11d9494089b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:49.967795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-39852091-eccc-432f-94a6-11d9494089b5 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:49.969043 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-39852091-eccc-432f-94a6-11d9494089b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:50.054335 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 556/1119] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:50.343777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538bb186-4be1-4bdb-a928-8f8843a73711 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:50.435113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:50.475832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538bb186-4be1-4bdb-a928-8f8843a73711 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:50.524287 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d39bdf-4e85-425f-addd-1645e7aafa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:50.525509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d39bdf-4e85-425f-addd-1645e7aafa69 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:50.525509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d39bdf-4e85-425f-addd-1645e7aafa69 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:50.525745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d39bdf-4e85-425f-addd-1645e7aafa69 None admin] PATH_INFO: `/v3/roles/be3ffa4db9954414868f116f97f3a929` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:50.527160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d39bdf-4e85-425f-addd-1645e7aafa69 None admin] RBAC: Authorizing `identity:get_role(role_id=be3ffa4db9954414868f116f97f3a929)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:50.528810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d39bdf-4e85-425f-addd-1645e7aafa69 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:50.530001 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 564/1120] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:24:50 2022] GET /identity/v3/roles/be3ffa4db9954414868f116f97f3a929 => generated 223 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:50.537010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39852091-eccc-432f-94a6-11d9494089b5 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:50.658513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39852091-eccc-432f-94a6-11d9494089b5 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:50.670504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:50.699071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028eaac-5fe4-42f1-b8f1-bd7e2e5204d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:50.699711 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028eaac-5fe4-42f1-b8f1-bd7e2e5204d9 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:50.700018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028eaac-5fe4-42f1-b8f1-bd7e2e5204d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:50.700298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028eaac-5fe4-42f1-b8f1-bd7e2e5204d9 None admin] PATH_INFO: `/v3/users/0bd45467d9c746cfaa40b7b0f7cdf44b` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:50.701615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028eaac-5fe4-42f1-b8f1-bd7e2e5204d9 None admin] RBAC: Authorizing `identity:get_user(user_id=0bd45467d9c746cfaa40b7b0f7cdf44b)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:50.703183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028eaac-5fe4-42f1-b8f1-bd7e2e5204d9 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:50.704869 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 557/1121] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:24:50 2022] GET /identity/v3/users/0bd45467d9c746cfaa40b7b0f7cdf44b => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:50.711488 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d39bdf-4e85-425f-addd-1645e7aafa69 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:50.834583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d39bdf-4e85-425f-addd-1645e7aafa69 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:50.879061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359a642-81a0-4f3b-bd57-fb6d26a5030d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:50.879698 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359a642-81a0-4f3b-bd57-fb6d26a5030d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:50.880005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359a642-81a0-4f3b-bd57-fb6d26a5030d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:50.880277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359a642-81a0-4f3b-bd57-fb6d26a5030d None admin] PATH_INFO: `/v3/projects/ac5674be19c04a06b5d0531f4c50b82f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:50.881513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359a642-81a0-4f3b-bd57-fb6d26a5030d None admin] RBAC: Authorizing `identity:get_project(project_id=ac5674be19c04a06b5d0531f4c50b82f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:50.882962 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359a642-81a0-4f3b-bd57-fb6d26a5030d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:50.884893 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 565/1122] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:24:50 2022] GET /identity/v3/projects/ac5674be19c04a06b5d0531f4c50b82f => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:50.891277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028eaac-5fe4-42f1-b8f1-bd7e2e5204d9 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:51.015843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028eaac-5fe4-42f1-b8f1-bd7e2e5204d9 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:51.051253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e12dbd-08d6-460a-b873-1bf7565cec48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:51.051932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e12dbd-08d6-460a-b873-1bf7565cec48 None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:51.052219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e12dbd-08d6-460a-b873-1bf7565cec48 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:51.052488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e12dbd-08d6-460a-b873-1bf7565cec48 None admin] PATH_INFO: `/v3/projects/ac5674be19c04a06b5d0531f4c50b82f/users/0bd45467d9c746cfaa40b7b0f7cdf44b/roles/be3ffa4db9954414868f116f97f3a929` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:51.055027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e12dbd-08d6-460a-b873-1bf7565cec48 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac5674be19c04a06b5d0531f4c50b82f, user_id=0bd45467d9c746cfaa40b7b0f7cdf44b, role_id=be3ffa4db9954414868f116f97f3a929)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:51.056778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e12dbd-08d6-460a-b873-1bf7565cec48 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:51.072919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 558/1123] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:24:50 2022] PUT /identity/v3/projects/ac5674be19c04a06b5d0531f4c50b82f/users/0bd45467d9c746cfaa40b7b0f7cdf44b/roles/be3ffa4db9954414868f116f97f3a929 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:24:52.925597 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc83a7c-e599-4921-900f-a296998f8a26 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:52.926517 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc83a7c-e599-4921-900f-a296998f8a26 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:52.926517 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc83a7c-e599-4921-900f-a296998f8a26 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:52.928723 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 566/1124] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:52.937846 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01c7ba-f484-44b7-9c79-0b5d38c38530 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:52.938517 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01c7ba-f484-44b7-9c79-0b5d38c38530 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:52.938639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01c7ba-f484-44b7-9c79-0b5d38c38530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:52.968289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-2c01c7ba-f484-44b7-9c79-0b5d38c38530 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:52.969478 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2c01c7ba-f484-44b7-9c79-0b5d38c38530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:53.004312 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:53.045578 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 559/1125] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:53.322834 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc83a7c-e599-4921-900f-a296998f8a26 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:53.329316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:53.442564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc83a7c-e599-4921-900f-a296998f8a26 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:53.482237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa67309-0bf3-476f-9018-aa1fa9b10356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:53.482880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa67309-0bf3-476f-9018-aa1fa9b10356 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:53.483151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa67309-0bf3-476f-9018-aa1fa9b10356 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:53.483418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa67309-0bf3-476f-9018-aa1fa9b10356 None admin] PATH_INFO: `/v3/roles/be3ffa4db9954414868f116f97f3a929` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:53.484893 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa67309-0bf3-476f-9018-aa1fa9b10356 None admin] RBAC: Authorizing `identity:get_role(role_id=be3ffa4db9954414868f116f97f3a929)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:53.486357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa67309-0bf3-476f-9018-aa1fa9b10356 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:53.487696 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 567/1126] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:24:53 2022] GET /identity/v3/roles/be3ffa4db9954414868f116f97f3a929 => generated 223 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:53.494830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01c7ba-f484-44b7-9c79-0b5d38c38530 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:53.623714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01c7ba-f484-44b7-9c79-0b5d38c38530 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:53.661650 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a9def-aed1-4361-af7c-b3f8153c332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:53.662280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9def-aed1-4361-af7c-b3f8153c332c None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:53.662552 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9def-aed1-4361-af7c-b3f8153c332c None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:53.662816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a9def-aed1-4361-af7c-b3f8153c332c None admin] PATH_INFO: `/v3/users/0bd45467d9c746cfaa40b7b0f7cdf44b` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:53.664083 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a9def-aed1-4361-af7c-b3f8153c332c None admin] RBAC: Authorizing `identity:get_user(user_id=0bd45467d9c746cfaa40b7b0f7cdf44b)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:53.665551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a9def-aed1-4361-af7c-b3f8153c332c None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:53.667191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 560/1127] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:24:53 2022] GET /identity/v3/users/0bd45467d9c746cfaa40b7b0f7cdf44b => generated 286 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:53.674243 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa67309-0bf3-476f-9018-aa1fa9b10356 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:53.753509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:53.791228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa67309-0bf3-476f-9018-aa1fa9b10356 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:53.826432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2e540-870f-422a-8a4b-f17f631094f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:53.827033 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e540-870f-422a-8a4b-f17f631094f3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:53.827317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e540-870f-422a-8a4b-f17f631094f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:53.827585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e540-870f-422a-8a4b-f17f631094f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:53.920319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2e540-870f-422a-8a4b-f17f631094f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:53.922464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2e540-870f-422a-8a4b-f17f631094f3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:53.997761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:54.040241 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 568/1128] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:24:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:24:54.049945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a9def-aed1-4361-af7c-b3f8153c332c None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:54.092251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:54.159196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a9def-aed1-4361-af7c-b3f8153c332c None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:54.194092 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcbdd7-12e1-42c8-8566-cf91a0eb1f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:54.194662 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcbdd7-12e1-42c8-8566-cf91a0eb1f05 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:54.194935 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcbdd7-12e1-42c8-8566-cf91a0eb1f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:54.195199 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcbdd7-12e1-42c8-8566-cf91a0eb1f05 None admin] PATH_INFO: `/v3/projects/ac5674be19c04a06b5d0531f4c50b82f` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:54.196506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcbdd7-12e1-42c8-8566-cf91a0eb1f05 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5674be19c04a06b5d0531f4c50b82f)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:54.197935 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcbdd7-12e1-42c8-8566-cf91a0eb1f05 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:54.199584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 561/1129] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:24:54 2022] GET /identity/v3/projects/ac5674be19c04a06b5d0531f4c50b82f => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:54.210133 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2e540-870f-422a-8a4b-f17f631094f3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:54.318409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2e540-870f-422a-8a4b-f17f631094f3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:54.354607 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513eac88-7b09-42b8-8a31-dfc0dce5d3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:54.355190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513eac88-7b09-42b8-8a31-dfc0dce5d3cb None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:54.355462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513eac88-7b09-42b8-8a31-dfc0dce5d3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:54.355734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513eac88-7b09-42b8-8a31-dfc0dce5d3cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:54.356481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513eac88-7b09-42b8-8a31-dfc0dce5d3cb None admin] RBAC: Adding query filter params (role.id=be3ffa4db9954414868f116f97f3a929, scope.project.id=ac5674be19c04a06b5d0531f4c50b82f, user.id=0bd45467d9c746cfaa40b7b0f7cdf44b) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:24:54.356786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513eac88-7b09-42b8-8a31-dfc0dce5d3cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:54.358203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513eac88-7b09-42b8-8a31-dfc0dce5d3cb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:54.364087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 569/1130] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:24:54 2022] GET /identity/v3/role_assignments?user.id=0bd45467d9c746cfaa40b7b0f7cdf44b&scope.project.id=ac5674be19c04a06b5d0531f4c50b82f&role.id=be3ffa4db9954414868f116f97f3a929 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:56.270068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962555df-0e09-499f-a6e7-438d61f93dcf None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:56.270617 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962555df-0e09-499f-a6e7-438d61f93dcf None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:56.270617 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962555df-0e09-499f-a6e7-438d61f93dcf None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:56.272680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 562/1131] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:56.279124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1072353-417f-4c17-9c98-5921e90bdda1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:56.279564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1072353-417f-4c17-9c98-5921e90bdda1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:56.279655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1072353-417f-4c17-9c98-5921e90bdda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:56.299769 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b1072353-417f-4c17-9c98-5921e90bdda1 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:56.300991 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b1072353-417f-4c17-9c98-5921e90bdda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:56.330867 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:56.370099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 570/1132] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:24:56.630804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962555df-0e09-499f-a6e7-438d61f93dcf None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:56.736664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962555df-0e09-499f-a6e7-438d61f93dcf None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:56.738957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:56.778126 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c2082-9121-4f5e-a3ff-36f2768eaf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:56.778771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c2082-9121-4f5e-a3ff-36f2768eaf36 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:56.779048 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c2082-9121-4f5e-a3ff-36f2768eaf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:56.779313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c2082-9121-4f5e-a3ff-36f2768eaf36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:56.780597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c2082-9121-4f5e-a3ff-36f2768eaf36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:56.782073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c2082-9121-4f5e-a3ff-36f2768eaf36 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:56.783680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 563/1133] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:56.789727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1072353-417f-4c17-9c98-5921e90bdda1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:56.897780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1072353-417f-4c17-9c98-5921e90bdda1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:56.932611 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c086f60-73ab-48b2-9a82-7a750e34bce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:56.933236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c086f60-73ab-48b2-9a82-7a750e34bce1 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:56.933510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c086f60-73ab-48b2-9a82-7a750e34bce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:56.933776 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c086f60-73ab-48b2-9a82-7a750e34bce1 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:56.934417 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c086f60-73ab-48b2-9a82-7a750e34bce1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:56.935844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c086f60-73ab-48b2-9a82-7a750e34bce1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:56.967538 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 571/1134] 10.222.0.62 () {60 vars in 1141 bytes} [Thu Sep 1 16:24:56 2022] POST /identity/v3/projects => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:24:58.905767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911b878-0054-4760-829d-bd5aec0c5c22 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:58.906357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911b878-0054-4760-829d-bd5aec0c5c22 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:58.906357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911b878-0054-4760-829d-bd5aec0c5c22 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:58.907641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 564/1135] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:58.916441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ab843-d293-4db8-a745-dfcd8c8ee73e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:58.916906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ab843-d293-4db8-a745-dfcd8c8ee73e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:58.917137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ab843-d293-4db8-a745-dfcd8c8ee73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:58.942458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b19ab843-d293-4db8-a745-dfcd8c8ee73e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:58.943662 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b19ab843-d293-4db8-a745-dfcd8c8ee73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:58.958449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:59.018514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 572/1136] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:24:59.297479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911b878-0054-4760-829d-bd5aec0c5c22 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:59.414055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911b878-0054-4760-829d-bd5aec0c5c22 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:59.450670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86081c0a-e440-417f-87e2-63a89901ad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:59.451294 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86081c0a-e440-417f-87e2-63a89901ad72 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:59.451563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86081c0a-e440-417f-87e2-63a89901ad72 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:59.451824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86081c0a-e440-417f-87e2-63a89901ad72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:59.453027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86081c0a-e440-417f-87e2-63a89901ad72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:59.454463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86081c0a-e440-417f-87e2-63a89901ad72 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:59.456073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 565/1137] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:24:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:24:59.463015 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ab843-d293-4db8-a745-dfcd8c8ee73e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:59.568498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ab843-d293-4db8-a745-dfcd8c8ee73e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:59.606245 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0576c0-635b-47db-8411-b45512e117ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:59.606847 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0576c0-635b-47db-8411-b45512e117ea None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:59.607118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0576c0-635b-47db-8411-b45512e117ea None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:59.607364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0576c0-635b-47db-8411-b45512e117ea None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:59.608037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0576c0-635b-47db-8411-b45512e117ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:59.609463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0576c0-635b-47db-8411-b45512e117ea None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:59.634008 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 573/1138] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:24:59 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:25:01.581675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b5a25-3855-4a91-8c93-784bf96339a1 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:01.582588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b5a25-3855-4a91-8c93-784bf96339a1 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:01.582684 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b5a25-3855-4a91-8c93-784bf96339a1 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:01.585778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 566/1139] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:25:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:01.591352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942c13a-1ae4-4496-872a-7861ae016ae1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:01.591766 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942c13a-1ae4-4496-872a-7861ae016ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:01.591880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942c13a-1ae4-4496-872a-7861ae016ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:01.600352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:01.612371 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8942c13a-1ae4-4496-872a-7861ae016ae1 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:01.613807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8942c13a-1ae4-4496-872a-7861ae016ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:01.683000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 574/1140] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:25:01.948325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b5a25-3855-4a91-8c93-784bf96339a1 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:02.003593 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:02.068851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b5a25-3855-4a91-8c93-784bf96339a1 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:02.114490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08d502-c850-4972-85d9-432f117542e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:02.115125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d502-c850-4972-85d9-432f117542e2 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:02.115397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d502-c850-4972-85d9-432f117542e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:02.115666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d502-c850-4972-85d9-432f117542e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:02.127391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08d502-c850-4972-85d9-432f117542e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-2b08d502-c850-4972-85d9-432f117542e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:25:02.132282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:25:02.133427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:25:02.136271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:25:02.136271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:25:02.136271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:25:02.136271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:25:02.136271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:25:02.136271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:25:02.136271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:25:02.136271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:25:02.136271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:25:02.136271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:25:02.136271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:25:02.136271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:25:02.136271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 567/1141] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:25:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:25:02.140531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942c13a-1ae4-4496-872a-7861ae016ae1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:02.251656 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942c13a-1ae4-4496-872a-7861ae016ae1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:02.286463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361d950-fd45-41da-95ad-463c5c34358c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:02.287065 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361d950-fd45-41da-95ad-463c5c34358c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:02.287340 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361d950-fd45-41da-95ad-463c5c34358c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:02.287609 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361d950-fd45-41da-95ad-463c5c34358c None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:02.288304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361d950-fd45-41da-95ad-463c5c34358c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:25:02.288606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361d950-fd45-41da-95ad-463c5c34358c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:02.289971 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361d950-fd45-41da-95ad-463c5c34358c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:02.291544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:02.301104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 575/1142] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:25:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:02.307531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08d502-c850-4972-85d9-432f117542e2 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:02.413670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:02.417407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08d502-c850-4972-85d9-432f117542e2 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:02.456992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51525982-bf70-43e4-8b16-1f79fbbbd605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:02.457617 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51525982-bf70-43e4-8b16-1f79fbbbd605 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:02.458205 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51525982-bf70-43e4-8b16-1f79fbbbd605 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:02.458472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51525982-bf70-43e4-8b16-1f79fbbbd605 None admin] PATH_INFO: `/v3/users/0545998f442048998da380493d01c01e` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:02.472119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51525982-bf70-43e4-8b16-1f79fbbbd605 None admin] RBAC: Authorizing `identity:get_user(user_id=0545998f442048998da380493d01c01e)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:02.473605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51525982-bf70-43e4-8b16-1f79fbbbd605 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:02.475218 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 568/1143] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:25:02 2022] GET /identity/v3/users/0545998f442048998da380493d01c01e => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:25:02.481433 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361d950-fd45-41da-95ad-463c5c34358c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:02.583250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361d950-fd45-41da-95ad-463c5c34358c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:02.621231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d25f8-5996-4d30-ae8b-cd3f24ceb375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:02.621851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d25f8-5996-4d30-ae8b-cd3f24ceb375 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:02.622079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d25f8-5996-4d30-ae8b-cd3f24ceb375 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:02.622341 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d25f8-5996-4d30-ae8b-cd3f24ceb375 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:02.684632 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:02.691413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d25f8-5996-4d30-ae8b-cd3f24ceb375 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:02.692886 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d25f8-5996-4d30-ae8b-cd3f24ceb375 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:02.805738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 576/1144] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:25:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:25:02.815082 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51525982-bf70-43e4-8b16-1f79fbbbd605 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:02.929230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51525982-bf70-43e4-8b16-1f79fbbbd605 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:02.967846 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361def28-50f1-4f22-af42-ada1c7e8933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:02.968516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361def28-50f1-4f22-af42-ada1c7e8933a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:02.968794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361def28-50f1-4f22-af42-ada1c7e8933a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:02.969058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361def28-50f1-4f22-af42-ada1c7e8933a None admin] PATH_INFO: `/v3/projects/bb5cede99f344caba94987ef9134db7c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:02.970308 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361def28-50f1-4f22-af42-ada1c7e8933a None admin] RBAC: Authorizing `identity:get_project(project_id=bb5cede99f344caba94987ef9134db7c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:02.971747 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361def28-50f1-4f22-af42-ada1c7e8933a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:02.973458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 569/1145] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:25:02 2022] GET /identity/v3/projects/bb5cede99f344caba94987ef9134db7c => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:02.980021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d25f8-5996-4d30-ae8b-cd3f24ceb375 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:03.082390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:03.090020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d25f8-5996-4d30-ae8b-cd3f24ceb375 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:03.137703 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f71875-65f4-4181-880c-85cfedf16699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:03.138304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f71875-65f4-4181-880c-85cfedf16699 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:03.138576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f71875-65f4-4181-880c-85cfedf16699 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:03.138844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f71875-65f4-4181-880c-85cfedf16699 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:03.139556 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f71875-65f4-4181-880c-85cfedf16699 None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=bb5cede99f344caba94987ef9134db7c, user.id=0545998f442048998da380493d01c01e) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:25:03.139861 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f71875-65f4-4181-880c-85cfedf16699 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:03.141310 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f71875-65f4-4181-880c-85cfedf16699 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:03.146769 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 577/1146] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:25:02 2022] GET /identity/v3/role_assignments?user.id=0545998f442048998da380493d01c01e&scope.project.id=bb5cede99f344caba94987ef9134db7c&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:05.025763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63afeb-0208-46ae-9c42-05bb94084748 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:05.026290 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63afeb-0208-46ae-9c42-05bb94084748 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:05.026290 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63afeb-0208-46ae-9c42-05bb94084748 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:05.027563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 570/1147] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:05.033977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e579908-e10c-4471-84db-6ec7313f829b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:05.034695 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e579908-e10c-4471-84db-6ec7313f829b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:05.034854 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e579908-e10c-4471-84db-6ec7313f829b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:05.062445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8e579908-e10c-4471-84db-6ec7313f829b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:05.063616 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8e579908-e10c-4471-84db-6ec7313f829b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:05.134449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 578/1148] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:25:05.398065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63afeb-0208-46ae-9c42-05bb94084748 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:05.428001 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:05.506987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63afeb-0208-46ae-9c42-05bb94084748 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:05.546346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feded06a-89f2-4ce9-9482-07dbe9d5a1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:05.546979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feded06a-89f2-4ce9-9482-07dbe9d5a1fb None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:05.547255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feded06a-89f2-4ce9-9482-07dbe9d5a1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:05.547528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feded06a-89f2-4ce9-9482-07dbe9d5a1fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:05.558519 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feded06a-89f2-4ce9-9482-07dbe9d5a1fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-feded06a-89f2-4ce9-9482-07dbe9d5a1fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:25:05.563403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:25:05.564395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:25:05.565255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:25:05.565255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:25:05.565255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:25:05.565255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:25:05.565255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:25:05.565255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:25:05.565255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:25:05.565255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:25:05.565255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:25:05.565255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:25:05.565255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:25:05.565255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:25:05.565255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 571/1149] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:25:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:25:05.573203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e579908-e10c-4471-84db-6ec7313f829b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:05.667976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:05.678498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e579908-e10c-4471-84db-6ec7313f829b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:05.713155 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064e541-e3ff-42e3-883c-f272aced0e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:05.713758 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064e541-e3ff-42e3-883c-f272aced0e0c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:05.714036 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064e541-e3ff-42e3-883c-f272aced0e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:05.714302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064e541-e3ff-42e3-883c-f272aced0e0c None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:05.714967 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064e541-e3ff-42e3-883c-f272aced0e0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:25:05.715266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064e541-e3ff-42e3-883c-f272aced0e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:05.716665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064e541-e3ff-42e3-883c-f272aced0e0c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:05.724704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 579/1150] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:25:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:25:05.733855 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feded06a-89f2-4ce9-9482-07dbe9d5a1fb None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:05.830281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:05.849023 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feded06a-89f2-4ce9-9482-07dbe9d5a1fb None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:05.885499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2288d5f-6594-4ab6-a850-2d1dee82debb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:05.886118 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2288d5f-6594-4ab6-a850-2d1dee82debb None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:05.886395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2288d5f-6594-4ab6-a850-2d1dee82debb None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:05.886659 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2288d5f-6594-4ab6-a850-2d1dee82debb None admin] PATH_INFO: `/v3/users/0545998f442048998da380493d01c01e` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:05.887819 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2288d5f-6594-4ab6-a850-2d1dee82debb None admin] RBAC: Authorizing `identity:get_user(user_id=0545998f442048998da380493d01c01e)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:05.889286 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2288d5f-6594-4ab6-a850-2d1dee82debb None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:05.890844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 572/1151] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:25:05 2022] GET /identity/v3/users/0545998f442048998da380493d01c01e => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:05.897882 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064e541-e3ff-42e3-883c-f272aced0e0c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:06.015449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064e541-e3ff-42e3-883c-f272aced0e0c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:06.055659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d2bf7-7ef0-4ee7-980b-150bfc75602e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:06.056460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d2bf7-7ef0-4ee7-980b-150bfc75602e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:06.056650 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d2bf7-7ef0-4ee7-980b-150bfc75602e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:06.056910 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d2bf7-7ef0-4ee7-980b-150bfc75602e None admin] PATH_INFO: `/v3/projects/bb5cede99f344caba94987ef9134db7c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:06.058199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d2bf7-7ef0-4ee7-980b-150bfc75602e None admin] RBAC: Authorizing `identity:get_project(project_id=bb5cede99f344caba94987ef9134db7c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:06.059624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d2bf7-7ef0-4ee7-980b-150bfc75602e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:06.061290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 580/1152] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:25:05 2022] GET /identity/v3/projects/bb5cede99f344caba94987ef9134db7c => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:25:06.071228 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2288d5f-6594-4ab6-a850-2d1dee82debb None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:06.181784 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2288d5f-6594-4ab6-a850-2d1dee82debb None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:06.224246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2079833-c2d1-425f-b9a9-5aa80cf22054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:06.224918 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2079833-c2d1-425f-b9a9-5aa80cf22054 None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:06.225187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2079833-c2d1-425f-b9a9-5aa80cf22054 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:06.225453 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2079833-c2d1-425f-b9a9-5aa80cf22054 None admin] PATH_INFO: `/v3/projects/bb5cede99f344caba94987ef9134db7c/users/0545998f442048998da380493d01c01e/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:06.227367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2079833-c2d1-425f-b9a9-5aa80cf22054 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb5cede99f344caba94987ef9134db7c, user_id=0545998f442048998da380493d01c01e, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:06.229525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2079833-c2d1-425f-b9a9-5aa80cf22054 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:06.245440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 573/1153] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:25:06 2022] PUT /identity/v3/projects/bb5cede99f344caba94987ef9134db7c/users/0545998f442048998da380493d01c01e/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:25:08.172339 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e97e2-f401-4261-8825-788865dd66af None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:08.172924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e97e2-f401-4261-8825-788865dd66af None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:08.172924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e97e2-f401-4261-8825-788865dd66af None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:08.174946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 581/1154] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:25:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:08.183801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100f8ab-eafb-4cb9-b51d-e482b0e4ab44 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:08.184227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100f8ab-eafb-4cb9-b51d-e482b0e4ab44 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:08.184434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100f8ab-eafb-4cb9-b51d-e482b0e4ab44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:08.210374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-7100f8ab-eafb-4cb9-b51d-e482b0e4ab44 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:08.211553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7100f8ab-eafb-4cb9-b51d-e482b0e4ab44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:08.221359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:08.285151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 574/1155] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:25:08.555508 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e97e2-f401-4261-8825-788865dd66af None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:08.581869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:08.671388 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e97e2-f401-4261-8825-788865dd66af None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:08.709482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169335e7-7c52-431c-bda5-cec6b98ae10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:08.710124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169335e7-7c52-431c-bda5-cec6b98ae10b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:08.710395 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169335e7-7c52-431c-bda5-cec6b98ae10b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:08.710656 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169335e7-7c52-431c-bda5-cec6b98ae10b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:08.723729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169335e7-7c52-431c-bda5-cec6b98ae10b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-169335e7-7c52-431c-bda5-cec6b98ae10b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:25:08.728584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:25:08.729552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:25:08.730453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:25:08.730453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:25:08.730453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:25:08.730453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:25:08.730453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:25:08.730453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:25:08.730453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:25:08.730453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:25:08.730453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:25:08.730453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:25:08.730453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:25:08.730453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:25:08.730453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 582/1156] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:25:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:25:08.736190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100f8ab-eafb-4cb9-b51d-e482b0e4ab44 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:08.841678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100f8ab-eafb-4cb9-b51d-e482b0e4ab44 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:08.877331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7fa14-2015-4f3d-bddd-65a78739e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:08.877940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7fa14-2015-4f3d-bddd-65a78739e40b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:08.878206 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7fa14-2015-4f3d-bddd-65a78739e40b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:08.878474 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7fa14-2015-4f3d-bddd-65a78739e40b None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:08.879130 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7fa14-2015-4f3d-bddd-65a78739e40b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:25:08.879427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7fa14-2015-4f3d-bddd-65a78739e40b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:08.880832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7fa14-2015-4f3d-bddd-65a78739e40b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:08.888833 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 575/1157] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:25:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:25:08.895106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169335e7-7c52-431c-bda5-cec6b98ae10b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:08.982597 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:09.002177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169335e7-7c52-431c-bda5-cec6b98ae10b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:09.039943 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ebf03-2ae7-4127-afe5-6eee621f6d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:09.041042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ebf03-2ae7-4127-afe5-6eee621f6d8f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:09.041537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ebf03-2ae7-4127-afe5-6eee621f6d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:09.041999 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ebf03-2ae7-4127-afe5-6eee621f6d8f None admin] PATH_INFO: `/v3/users/0545998f442048998da380493d01c01e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:09.044012 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ebf03-2ae7-4127-afe5-6eee621f6d8f None admin] RBAC: Authorizing `identity:get_user(user_id=0545998f442048998da380493d01c01e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:09.046756 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ebf03-2ae7-4127-afe5-6eee621f6d8f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:09.049453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 583/1158] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:25:08 2022] GET /identity/v3/users/0545998f442048998da380493d01c01e => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:09.056319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7fa14-2015-4f3d-bddd-65a78739e40b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:09.071962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:09.164602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7fa14-2015-4f3d-bddd-65a78739e40b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:09.199928 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cacf0dd-63a9-40bc-8b49-21a24f66caf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:09.200542 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacf0dd-63a9-40bc-8b49-21a24f66caf7 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:09.200794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacf0dd-63a9-40bc-8b49-21a24f66caf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:09.201063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacf0dd-63a9-40bc-8b49-21a24f66caf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:09.272741 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cacf0dd-63a9-40bc-8b49-21a24f66caf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:09.274205 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cacf0dd-63a9-40bc-8b49-21a24f66caf7 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:09.281985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:09.391600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 576/1159] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:25:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:25:09.400713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ebf03-2ae7-4127-afe5-6eee621f6d8f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:09.507713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ebf03-2ae7-4127-afe5-6eee621f6d8f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:09.542100 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ebb97-d03b-4b4c-a7d1-eb9f3c13870a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:09.542710 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ebb97-d03b-4b4c-a7d1-eb9f3c13870a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:09.542978 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ebb97-d03b-4b4c-a7d1-eb9f3c13870a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:09.543239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ebb97-d03b-4b4c-a7d1-eb9f3c13870a None admin] PATH_INFO: `/v3/projects/bb5cede99f344caba94987ef9134db7c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:09.544366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ebb97-d03b-4b4c-a7d1-eb9f3c13870a None admin] RBAC: Authorizing `identity:get_project(project_id=bb5cede99f344caba94987ef9134db7c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:09.545785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ebb97-d03b-4b4c-a7d1-eb9f3c13870a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:09.547401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 584/1160] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:25:09 2022] GET /identity/v3/projects/bb5cede99f344caba94987ef9134db7c => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:09.554485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cacf0dd-63a9-40bc-8b49-21a24f66caf7 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:09.656646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cacf0dd-63a9-40bc-8b49-21a24f66caf7 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:09.663970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:09.704953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87b032-57a7-42ac-9517-08385e2c69d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:09.705568 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87b032-57a7-42ac-9517-08385e2c69d9 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:09.705848 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87b032-57a7-42ac-9517-08385e2c69d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:09.706118 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87b032-57a7-42ac-9517-08385e2c69d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:09.706841 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87b032-57a7-42ac-9517-08385e2c69d9 None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=bb5cede99f344caba94987ef9134db7c, user.id=0545998f442048998da380493d01c01e) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:25:09.707146 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87b032-57a7-42ac-9517-08385e2c69d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:09.708956 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87b032-57a7-42ac-9517-08385e2c69d9 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:09.714871 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 577/1161] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:25:09 2022] GET /identity/v3/role_assignments?user.id=0545998f442048998da380493d01c01e&scope.project.id=bb5cede99f344caba94987ef9134db7c&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:11.634124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bde55-0a35-4b4d-82c3-33622a543e2f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:11.634753 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bde55-0a35-4b4d-82c3-33622a543e2f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:11.634753 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bde55-0a35-4b4d-82c3-33622a543e2f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:11.636186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 585/1162] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:11.641782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ef369-5a86-41a1-9365-f5c9732a994f None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:11.642199 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ef369-5a86-41a1-9365-f5c9732a994f None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:11.642288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ef369-5a86-41a1-9365-f5c9732a994f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:11.662450 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e45ef369-5a86-41a1-9365-f5c9732a994f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:11.663641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e45ef369-5a86-41a1-9365-f5c9732a994f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:11.735719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 578/1163] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:25:11.999655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bde55-0a35-4b4d-82c3-33622a543e2f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:12.023208 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:12.119228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bde55-0a35-4b4d-82c3-33622a543e2f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:12.157021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b5281-432d-4094-94af-9a601aabd685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:12.157858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b5281-432d-4094-94af-9a601aabd685 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:12.158204 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b5281-432d-4094-94af-9a601aabd685 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:12.158564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b5281-432d-4094-94af-9a601aabd685 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:12.231641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b5281-432d-4094-94af-9a601aabd685 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:12.233148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b5281-432d-4094-94af-9a601aabd685 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:12.250629 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:12.344412 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 586/1164] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:25:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:25:12.350845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ef369-5a86-41a1-9365-f5c9732a994f None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:12.456968 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:12.460882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ef369-5a86-41a1-9365-f5c9732a994f None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:12.495728 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f9a25-9431-43b1-81aa-4d9c7a357cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:12.496385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f9a25-9431-43b1-81aa-4d9c7a357cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:12.496655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f9a25-9431-43b1-81aa-4d9c7a357cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:12.496915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f9a25-9431-43b1-81aa-4d9c7a357cb1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:12.504589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f9a25-9431-43b1-81aa-4d9c7a357cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:12.505974 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f9a25-9431-43b1-81aa-4d9c7a357cb1 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-a72f9a25-9431-43b1-81aa-4d9c7a357cb1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:25:12.518245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:25:12.519390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:25:12.521249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 579/1165] 10.222.0.62 () {56 vars in 1117 bytes} [Thu Sep 1 16:25:12 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:25:12.527734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b5281-432d-4094-94af-9a601aabd685 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:12.629572 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b5281-432d-4094-94af-9a601aabd685 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:12.646647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:12.667911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0836c6-db7a-4c47-b4ce-bab8050477de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:12.668506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0836c6-db7a-4c47-b4ce-bab8050477de None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:12.668811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0836c6-db7a-4c47-b4ce-bab8050477de None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:12.669043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0836c6-db7a-4c47-b4ce-bab8050477de None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:12.669655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0836c6-db7a-4c47-b4ce-bab8050477de None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:25:12.669950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0836c6-db7a-4c47-b4ce-bab8050477de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:12.671305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0836c6-db7a-4c47-b4ce-bab8050477de None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:12.678249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 587/1166] 10.222.0.62 () {56 vars in 1115 bytes} [Thu Sep 1 16:25:12 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:14.556046 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a744c9-4383-4dc1-88aa-eed2dc0b950a None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:14.557007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a744c9-4383-4dc1-88aa-eed2dc0b950a None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:14.557305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a744c9-4383-4dc1-88aa-eed2dc0b950a None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:14.559579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 580/1167] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:25:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:14.568727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90834f-eb44-455f-b0f1-c50a7be6957d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:14.569447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90834f-eb44-455f-b0f1-c50a7be6957d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:14.569632 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90834f-eb44-455f-b0f1-c50a7be6957d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:14.596510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2c90834f-eb44-455f-b0f1-c50a7be6957d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:14.597719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2c90834f-eb44-455f-b0f1-c50a7be6957d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:14.667085 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 588/1168] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:25:14.937576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a744c9-4383-4dc1-88aa-eed2dc0b950a None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:15.057768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a744c9-4383-4dc1-88aa-eed2dc0b950a None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:15.095300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37077bfe-9323-4425-a6d3-42f32137cdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:15.095925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37077bfe-9323-4425-a6d3-42f32137cdd5 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:15.096212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37077bfe-9323-4425-a6d3-42f32137cdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:15.096479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37077bfe-9323-4425-a6d3-42f32137cdd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:15.097141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37077bfe-9323-4425-a6d3-42f32137cdd5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:15.098519 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37077bfe-9323-4425-a6d3-42f32137cdd5 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:15.126653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 581/1169] 10.222.0.62 () {60 vars in 1138 bytes} [Thu Sep 1 16:25:14 2022] POST /identity/v3/domains => generated 267 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:25:17.031699 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c10b6-85a1-4458-9377-745bc04c3848 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:17.032732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c10b6-85a1-4458-9377-745bc04c3848 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:17.032732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c10b6-85a1-4458-9377-745bc04c3848 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:17.034714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 589/1170] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:25:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:17.040154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e21c4-5d4f-4f41-b262-28ce7687817b None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:17.040597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e21c4-5d4f-4f41-b262-28ce7687817b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:17.040837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e21c4-5d4f-4f41-b262-28ce7687817b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:17.060853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6a8e21c4-5d4f-4f41-b262-28ce7687817b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:17.062068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6a8e21c4-5d4f-4f41-b262-28ce7687817b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:17.076508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:17.140800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 582/1171] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:25:17.416012 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c10b6-85a1-4458-9377-745bc04c3848 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:17.542785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c10b6-85a1-4458-9377-745bc04c3848 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:17.553653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:17.578728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447afb1b-8b1c-4d92-9144-f539b7f2a9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:17.579363 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447afb1b-8b1c-4d92-9144-f539b7f2a9dc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:17.579635 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447afb1b-8b1c-4d92-9144-f539b7f2a9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:17.579922 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447afb1b-8b1c-4d92-9144-f539b7f2a9dc None admin] PATH_INFO: `/v3/domains/db4f91b2fb3f46679385085370301b94` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:17.591780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447afb1b-8b1c-4d92-9144-f539b7f2a9dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=db4f91b2fb3f46679385085370301b94)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:17.593247 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447afb1b-8b1c-4d92-9144-f539b7f2a9dc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:17.594836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 590/1172] 10.222.0.62 () {56 vars in 1183 bytes} [Thu Sep 1 16:25:17 2022] GET /identity/v3/domains/db4f91b2fb3f46679385085370301b94 => generated 267 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:17.601796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e21c4-5d4f-4f41-b262-28ce7687817b None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:17.707186 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e21c4-5d4f-4f41-b262-28ce7687817b None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:17.744332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b916329-1cdb-472e-a3ff-bcac9370a80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:17.744939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b916329-1cdb-472e-a3ff-bcac9370a80f None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:17.745202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b916329-1cdb-472e-a3ff-bcac9370a80f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:17.745488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b916329-1cdb-472e-a3ff-bcac9370a80f None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:17.746149 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b916329-1cdb-472e-a3ff-bcac9370a80f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:17.747580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b916329-1cdb-472e-a3ff-bcac9370a80f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:17.780023 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 583/1173] 10.222.0.62 () {60 vars in 1141 bytes} [Thu Sep 1 16:25:17 2022] POST /identity/v3/projects => generated 361 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:25:19.762932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11107b14-2ccb-4571-b0d4-02c33d22fb6a None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:19.763639 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11107b14-2ccb-4571-b0d4-02c33d22fb6a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:19.763756 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11107b14-2ccb-4571-b0d4-02c33d22fb6a None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:19.766156 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 591/1174] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:25:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:19.774209 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c428034-af15-4cf4-9b93-8b51afe750a4 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:19.774744 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c428034-af15-4cf4-9b93-8b51afe750a4 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:19.775124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c428034-af15-4cf4-9b93-8b51afe750a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:19.782294 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:19.796273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-0c428034-af15-4cf4-9b93-8b51afe750a4 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:19.797479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0c428034-af15-4cf4-9b93-8b51afe750a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:19.869780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 584/1175] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:25:20.135268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11107b14-2ccb-4571-b0d4-02c33d22fb6a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:20.246192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11107b14-2ccb-4571-b0d4-02c33d22fb6a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:20.286813 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3295856-8923-4c86-8776-caeacf2b1a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:20.287447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3295856-8923-4c86-8776-caeacf2b1a63 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:20.287712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3295856-8923-4c86-8776-caeacf2b1a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:20.288005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3295856-8923-4c86-8776-caeacf2b1a63 None admin] PATH_INFO: `/v3/domains/db4f91b2fb3f46679385085370301b94` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:20.289262 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3295856-8923-4c86-8776-caeacf2b1a63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=db4f91b2fb3f46679385085370301b94)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:20.290761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3295856-8923-4c86-8776-caeacf2b1a63 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:20.292344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 592/1176] 10.222.0.62 () {56 vars in 1183 bytes} [Thu Sep 1 16:25:20 2022] GET /identity/v3/domains/db4f91b2fb3f46679385085370301b94 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:20.298940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c428034-af15-4cf4-9b93-8b51afe750a4 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:20.418372 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c428034-af15-4cf4-9b93-8b51afe750a4 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:20.447475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:20.453872 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4828829-05ce-4b66-9ee2-c2e3a519d3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:20.454482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828829-05ce-4b66-9ee2-c2e3a519d3e7 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:20.454755 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828829-05ce-4b66-9ee2-c2e3a519d3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:20.455017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4828829-05ce-4b66-9ee2-c2e3a519d3e7 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:20.455649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4828829-05ce-4b66-9ee2-c2e3a519d3e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:20.457138 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4828829-05ce-4b66-9ee2-c2e3a519d3e7 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:20.483180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 585/1177] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:25:20 2022] POST /identity/v3/users => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:25:22.464030 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6cc2c-a396-45a7-9b60-ca438fcde6ee None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:22.464892 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6cc2c-a396-45a7-9b60-ca438fcde6ee None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:22.464892 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6cc2c-a396-45a7-9b60-ca438fcde6ee None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:22.467046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 593/1178] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:25:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:22.473400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d49aa-f979-476c-8ff5-9462c662a3b4 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:22.474166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d49aa-f979-476c-8ff5-9462c662a3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:22.474285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d49aa-f979-476c-8ff5-9462c662a3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:22.497482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-ec8d49aa-f979-476c-8ff5-9462c662a3b4 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:22.498717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ec8d49aa-f979-476c-8ff5-9462c662a3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:22.571830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 586/1179] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:25:22.852666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6cc2c-a396-45a7-9b60-ca438fcde6ee None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:22.903305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:22.970412 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6cc2c-a396-45a7-9b60-ca438fcde6ee None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:23.005692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106e56f-7894-4f3b-8e6d-61f052da6cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:23.006333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106e56f-7894-4f3b-8e6d-61f052da6cff None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:23.006605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106e56f-7894-4f3b-8e6d-61f052da6cff None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:23.006868 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106e56f-7894-4f3b-8e6d-61f052da6cff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:23.017698 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106e56f-7894-4f3b-8e6d-61f052da6cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-4106e56f-7894-4f3b-8e6d-61f052da6cff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:25:23.022370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:25:23.023484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:25:23.024528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:25:23.024528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:25:23.024528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:25:23.024528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:25:23.024528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:25:23.024528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:25:23.024528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:25:23.024528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:25:23.024528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:25:23.024528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:25:23.024528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:25:23.024528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:25:23.024528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 594/1180] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:25:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:25:23.029784 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d49aa-f979-476c-8ff5-9462c662a3b4 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:23.141570 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d49aa-f979-476c-8ff5-9462c662a3b4 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:23.159975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:23.179961 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2c423-6059-479f-a2b1-96673767dea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:23.180587 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2c423-6059-479f-a2b1-96673767dea7 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:23.180864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2c423-6059-479f-a2b1-96673767dea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:23.181131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2c423-6059-479f-a2b1-96673767dea7 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:23.181800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2c423-6059-479f-a2b1-96673767dea7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:25:23.182515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2c423-6059-479f-a2b1-96673767dea7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:23.184415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2c423-6059-479f-a2b1-96673767dea7 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:23.193001 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 587/1181] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:25:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:23.199916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106e56f-7894-4f3b-8e6d-61f052da6cff None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:23.312077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:23.315931 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106e56f-7894-4f3b-8e6d-61f052da6cff None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:23.355687 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38b7ac-094d-4b9c-a485-bb81009f918b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:23.356334 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38b7ac-094d-4b9c-a485-bb81009f918b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:23.356601 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38b7ac-094d-4b9c-a485-bb81009f918b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:23.356864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38b7ac-094d-4b9c-a485-bb81009f918b None admin] PATH_INFO: `/v3/users/1d8d98c9c959406d981b718aa09f8a3d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:23.371945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38b7ac-094d-4b9c-a485-bb81009f918b None admin] RBAC: Authorizing `identity:get_user(user_id=1d8d98c9c959406d981b718aa09f8a3d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:23.373387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38b7ac-094d-4b9c-a485-bb81009f918b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:23.374977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 595/1182] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:25:23 2022] GET /identity/v3/users/1d8d98c9c959406d981b718aa09f8a3d => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:23.385011 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2c423-6059-479f-a2b1-96673767dea7 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:23.495899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2c423-6059-479f-a2b1-96673767dea7 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:23.534762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e365a7-21f6-41f6-8d6b-3785d3c444a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:23.535360 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e365a7-21f6-41f6-8d6b-3785d3c444a4 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:23.535637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e365a7-21f6-41f6-8d6b-3785d3c444a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:23.535931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e365a7-21f6-41f6-8d6b-3785d3c444a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:23.585932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:23.610312 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e365a7-21f6-41f6-8d6b-3785d3c444a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:23.612756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e365a7-21f6-41f6-8d6b-3785d3c444a4 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:23.724686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 588/1183] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:25:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:25:23.731800 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38b7ac-094d-4b9c-a485-bb81009f918b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:23.854278 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38b7ac-094d-4b9c-a485-bb81009f918b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:23.890539 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab776b5-0de6-4baa-b779-5fd3a0a4eb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:23.891160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab776b5-0de6-4baa-b779-5fd3a0a4eb21 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:23.891425 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab776b5-0de6-4baa-b779-5fd3a0a4eb21 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:23.891690 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab776b5-0de6-4baa-b779-5fd3a0a4eb21 None admin] PATH_INFO: `/v3/projects/8dd28935cb414720aabc9801213d3bd9` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:23.892890 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab776b5-0de6-4baa-b779-5fd3a0a4eb21 None admin] RBAC: Authorizing `identity:get_project(project_id=8dd28935cb414720aabc9801213d3bd9)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:23.894313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab776b5-0de6-4baa-b779-5fd3a0a4eb21 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:23.896072 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 596/1184] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:25:23 2022] GET /identity/v3/projects/8dd28935cb414720aabc9801213d3bd9 => generated 361 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:23.902604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e365a7-21f6-41f6-8d6b-3785d3c444a4 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:23.987696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:24.007592 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e365a7-21f6-41f6-8d6b-3785d3c444a4 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:24.050693 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7ebcf-ba71-4145-b6ad-f3e5592ce64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:24.051217 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7ebcf-ba71-4145-b6ad-f3e5592ce64a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:24.051489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7ebcf-ba71-4145-b6ad-f3e5592ce64a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:24.051752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7ebcf-ba71-4145-b6ad-f3e5592ce64a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:24.052516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7ebcf-ba71-4145-b6ad-f3e5592ce64a None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=8dd28935cb414720aabc9801213d3bd9, user.id=1d8d98c9c959406d981b718aa09f8a3d) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:25:24.052816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7ebcf-ba71-4145-b6ad-f3e5592ce64a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:24.054467 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7ebcf-ba71-4145-b6ad-f3e5592ce64a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:24.060749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 589/1185] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:25:23 2022] GET /identity/v3/role_assignments?user.id=1d8d98c9c959406d981b718aa09f8a3d&scope.project.id=8dd28935cb414720aabc9801213d3bd9&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:25.908385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e36114-c1da-47d3-812c-30b6084dfca0 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:25.909244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e36114-c1da-47d3-812c-30b6084dfca0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:25.909404 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e36114-c1da-47d3-812c-30b6084dfca0 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:25.911454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 597/1186] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:25:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:25.919760 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ae439-5e24-4837-899e-a589c78a87c5 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:25.920243 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ae439-5e24-4837-899e-a589c78a87c5 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:25.920340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ae439-5e24-4837-899e-a589c78a87c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:25.939724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-8e7ae439-5e24-4837-899e-a589c78a87c5 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:25.940932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8e7ae439-5e24-4837-899e-a589c78a87c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:26.018283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 590/1187] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:25:26.285277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e36114-c1da-47d3-812c-30b6084dfca0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:26.321164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:26.418926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e36114-c1da-47d3-812c-30b6084dfca0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:26.474983 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbdaac-c91e-444b-aea5-fe27fd0a091b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:26.475608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbdaac-c91e-444b-aea5-fe27fd0a091b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:26.475878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbdaac-c91e-444b-aea5-fe27fd0a091b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:26.476182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbdaac-c91e-444b-aea5-fe27fd0a091b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:26.487271 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbdaac-c91e-444b-aea5-fe27fd0a091b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-93fbdaac-c91e-444b-aea5-fe27fd0a091b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:25:26.492127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:25:26.493185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:25:26.494663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:25:26.494663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:25:26.494663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:25:26.494663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:25:26.494663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:25:26.494663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:25:26.494663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:25:26.494663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:25:26.494663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:25:26.494663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:25:26.494663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:25:26.494663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:25:26.494663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 598/1188] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:25:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:25:26.500242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ae439-5e24-4837-899e-a589c78a87c5 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:26.583961 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:26.610803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ae439-5e24-4837-899e-a589c78a87c5 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:26.645942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951ab3b-6a27-4650-b9d5-531181910054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:26.646529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951ab3b-6a27-4650-b9d5-531181910054 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:26.646799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951ab3b-6a27-4650-b9d5-531181910054 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:26.647061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951ab3b-6a27-4650-b9d5-531181910054 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:26.647714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951ab3b-6a27-4650-b9d5-531181910054 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:25:26.648035 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951ab3b-6a27-4650-b9d5-531181910054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:26.649401 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951ab3b-6a27-4650-b9d5-531181910054 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:26.657571 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 591/1189] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:25:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:26.663628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbdaac-c91e-444b-aea5-fe27fd0a091b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:26.751615 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:26.770834 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbdaac-c91e-444b-aea5-fe27fd0a091b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:26.808382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d2626-66b0-4b80-9336-003fd0913f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:26.809137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d2626-66b0-4b80-9336-003fd0913f17 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:26.809416 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d2626-66b0-4b80-9336-003fd0913f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:26.809685 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d2626-66b0-4b80-9336-003fd0913f17 None admin] PATH_INFO: `/v3/users/1d8d98c9c959406d981b718aa09f8a3d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:26.810899 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d2626-66b0-4b80-9336-003fd0913f17 None admin] RBAC: Authorizing `identity:get_user(user_id=1d8d98c9c959406d981b718aa09f8a3d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:26.812349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d2626-66b0-4b80-9336-003fd0913f17 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:26.813913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 599/1190] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:25:26 2022] GET /identity/v3/users/1d8d98c9c959406d981b718aa09f8a3d => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:26.823095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951ab3b-6a27-4650-b9d5-531181910054 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:26.932086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951ab3b-6a27-4650-b9d5-531181910054 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:26.970781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4d4ab-d02b-4949-acc7-0e3342e9226c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:26.971404 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4d4ab-d02b-4949-acc7-0e3342e9226c None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:26.971679 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4d4ab-d02b-4949-acc7-0e3342e9226c None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:26.971951 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4d4ab-d02b-4949-acc7-0e3342e9226c None admin] PATH_INFO: `/v3/projects/8dd28935cb414720aabc9801213d3bd9` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:26.973217 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4d4ab-d02b-4949-acc7-0e3342e9226c None admin] RBAC: Authorizing `identity:get_project(project_id=8dd28935cb414720aabc9801213d3bd9)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:26.974675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4d4ab-d02b-4949-acc7-0e3342e9226c None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:26.976340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 592/1191] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:25:26 2022] GET /identity/v3/projects/8dd28935cb414720aabc9801213d3bd9 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:26.983699 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d2626-66b0-4b80-9336-003fd0913f17 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:27.115172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d2626-66b0-4b80-9336-003fd0913f17 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:27.155869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171decdd-d898-4610-857d-0a688db1da2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:27.156553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171decdd-d898-4610-857d-0a688db1da2d None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:27.156832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171decdd-d898-4610-857d-0a688db1da2d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:27.157099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171decdd-d898-4610-857d-0a688db1da2d None admin] PATH_INFO: `/v3/projects/8dd28935cb414720aabc9801213d3bd9/users/1d8d98c9c959406d981b718aa09f8a3d/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:27.159272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171decdd-d898-4610-857d-0a688db1da2d None admin] RBAC: Authorizing `identity:create_grant(project_id=8dd28935cb414720aabc9801213d3bd9, user_id=1d8d98c9c959406d981b718aa09f8a3d, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:27.160756 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171decdd-d898-4610-857d-0a688db1da2d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:27.175583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 600/1192] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:25:26 2022] PUT /identity/v3/projects/8dd28935cb414720aabc9801213d3bd9/users/1d8d98c9c959406d981b718aa09f8a3d/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:25:29.043773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564845a0-06c3-48fd-8b10-90c55310d105 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:29.044302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564845a0-06c3-48fd-8b10-90c55310d105 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:29.044302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564845a0-06c3-48fd-8b10-90c55310d105 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:29.046320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 593/1193] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:25:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:29.054970 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1bfe2-d647-429d-a4c5-0f08499f7344 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:29.055356 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1bfe2-d647-429d-a4c5-0f08499f7344 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:29.055590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1bfe2-d647-429d-a4c5-0f08499f7344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:29.081915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-34a1bfe2-d647-429d-a4c5-0f08499f7344 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:29.083429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-34a1bfe2-d647-429d-a4c5-0f08499f7344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:29.091806 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:29.156687 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 601/1194] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:25:29.447390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564845a0-06c3-48fd-8b10-90c55310d105 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:29.453784 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:29.565614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564845a0-06c3-48fd-8b10-90c55310d105 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:29.600490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27243e6-4fd7-4052-87c1-e05a4269f0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:29.601106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27243e6-4fd7-4052-87c1-e05a4269f0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:29.601370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27243e6-4fd7-4052-87c1-e05a4269f0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:29.601637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27243e6-4fd7-4052-87c1-e05a4269f0c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:29.613014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27243e6-4fd7-4052-87c1-e05a4269f0c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-d27243e6-4fd7-4052-87c1-e05a4269f0c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:25:29.617787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:25:29.618887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:25:29.620169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:25:29.620169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:25:29.620169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:25:29.620169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:25:29.620169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:25:29.620169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:25:29.620169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:25:29.620169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:25:29.620169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:25:29.620169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:25:29.620169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:25:29.620169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:25:29.620169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 594/1195] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:25:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:25:29.625195 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1bfe2-d647-429d-a4c5-0f08499f7344 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:29.728586 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1bfe2-d647-429d-a4c5-0f08499f7344 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:29.763129 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f72a63-5801-4bd6-a74b-85112ef7d59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:29.763735 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f72a63-5801-4bd6-a74b-85112ef7d59f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:29.764056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f72a63-5801-4bd6-a74b-85112ef7d59f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:29.764321 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f72a63-5801-4bd6-a74b-85112ef7d59f None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:29.764969 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f72a63-5801-4bd6-a74b-85112ef7d59f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:25:29.765278 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f72a63-5801-4bd6-a74b-85112ef7d59f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:29.766659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f72a63-5801-4bd6-a74b-85112ef7d59f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:29.774872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 602/1196] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:25:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:29.780949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27243e6-4fd7-4052-87c1-e05a4269f0c8 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:29.845724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:29.884981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27243e6-4fd7-4052-87c1-e05a4269f0c8 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:29.920170 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb3ace-f585-4da9-bb22-7a13333b3f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:29.920819 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb3ace-f585-4da9-bb22-7a13333b3f25 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:29.921088 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb3ace-f585-4da9-bb22-7a13333b3f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:29.921347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb3ace-f585-4da9-bb22-7a13333b3f25 None admin] PATH_INFO: `/v3/users/1d8d98c9c959406d981b718aa09f8a3d` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:29.922579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb3ace-f585-4da9-bb22-7a13333b3f25 None admin] RBAC: Authorizing `identity:get_user(user_id=1d8d98c9c959406d981b718aa09f8a3d)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:29.924034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb3ace-f585-4da9-bb22-7a13333b3f25 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:29.925625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 595/1197] 10.222.0.62 () {56 vars in 1177 bytes} [Thu Sep 1 16:25:29 2022] GET /identity/v3/users/1d8d98c9c959406d981b718aa09f8a3d => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:29.931857 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f72a63-5801-4bd6-a74b-85112ef7d59f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:29.936039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:30.040225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f72a63-5801-4bd6-a74b-85112ef7d59f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:30.077891 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f6384-9afc-4e17-aa47-b3dc72910ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:30.078528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f6384-9afc-4e17-aa47-b3dc72910ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:30.078787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f6384-9afc-4e17-aa47-b3dc72910ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:30.079041 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f6384-9afc-4e17-aa47-b3dc72910ca4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:30.147678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f6384-9afc-4e17-aa47-b3dc72910ca4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:30.149311 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f6384-9afc-4e17-aa47-b3dc72910ca4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:30.152665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:30.258657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 603/1198] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:25:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:25:30.266064 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb3ace-f585-4da9-bb22-7a13333b3f25 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:30.385296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb3ace-f585-4da9-bb22-7a13333b3f25 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:30.415541 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:30.421940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7b8f6-f871-4a24-98eb-02ae42517f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:30.422632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7b8f6-f871-4a24-98eb-02ae42517f6b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:30.422917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7b8f6-f871-4a24-98eb-02ae42517f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:30.423179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7b8f6-f871-4a24-98eb-02ae42517f6b None admin] PATH_INFO: `/v3/projects/8dd28935cb414720aabc9801213d3bd9` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:30.424460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7b8f6-f871-4a24-98eb-02ae42517f6b None admin] RBAC: Authorizing `identity:get_project(project_id=8dd28935cb414720aabc9801213d3bd9)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:30.426010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7b8f6-f871-4a24-98eb-02ae42517f6b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:30.427801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 596/1199] 10.222.0.62 () {56 vars in 1186 bytes} [Thu Sep 1 16:25:30 2022] GET /identity/v3/projects/8dd28935cb414720aabc9801213d3bd9 => generated 361 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:30.435359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f6384-9afc-4e17-aa47-b3dc72910ca4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:30.549124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f6384-9afc-4e17-aa47-b3dc72910ca4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:30.551499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:30.593901 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0837ab-0fdd-4009-b6b4-0fc7a9c1410c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:30.594500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0837ab-0fdd-4009-b6b4-0fc7a9c1410c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:30.594775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0837ab-0fdd-4009-b6b4-0fc7a9c1410c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:30.595044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0837ab-0fdd-4009-b6b4-0fc7a9c1410c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:30.595769 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0837ab-0fdd-4009-b6b4-0fc7a9c1410c None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=8dd28935cb414720aabc9801213d3bd9, user.id=1d8d98c9c959406d981b718aa09f8a3d) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:25:30.596108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0837ab-0fdd-4009-b6b4-0fc7a9c1410c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:30.597542 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0837ab-0fdd-4009-b6b4-0fc7a9c1410c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:30.603532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 604/1200] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:25:30 2022] GET /identity/v3/role_assignments?user.id=1d8d98c9c959406d981b718aa09f8a3d&scope.project.id=8dd28935cb414720aabc9801213d3bd9&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:25:40.600839 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=16 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:26.620282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd84765-6c16-40ad-87d2-09446ca69af3 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:26.621014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd84765-6c16-40ad-87d2-09446ca69af3 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:26.621014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd84765-6c16-40ad-87d2-09446ca69af3 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:26.622886 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 597/1201] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:26:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:26.628755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9426118-9caf-4a10-91fd-145865a6c13a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:26.629295 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9426118-9caf-4a10-91fd-145865a6c13a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:26.629295 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9426118-9caf-4a10-91fd-145865a6c13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:26.649524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c9426118-9caf-4a10-91fd-145865a6c13a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:26.650779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c9426118-9caf-4a10-91fd-145865a6c13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:26.737190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 605/1202] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:26:27.054596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd84765-6c16-40ad-87d2-09446ca69af3 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:27.186950 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd84765-6c16-40ad-87d2-09446ca69af3 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:27.231921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc73d17-e5f6-4ca0-b1ab-abf781db0a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:27.233011 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc73d17-e5f6-4ca0-b1ab-abf781db0a3c None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:27.233504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc73d17-e5f6-4ca0-b1ab-abf781db0a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:27.233974 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc73d17-e5f6-4ca0-b1ab-abf781db0a3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:27.252509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc73d17-e5f6-4ca0-b1ab-abf781db0a3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:27.254062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc73d17-e5f6-4ca0-b1ab-abf781db0a3c None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-4bc73d17-e5f6-4ca0-b1ab-abf781db0a3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:26:27.271193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:27.272509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:26:27.274228 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:26:27.274228 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:27.274228 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:27.274228 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 598/1203] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:27.283333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9426118-9caf-4a10-91fd-145865a6c13a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:27.413938 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9426118-9caf-4a10-91fd-145865a6c13a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:27.446830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:27.453298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8116d9c-8fc8-4489-b6ec-189b34607cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:27.454182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8116d9c-8fc8-4489-b6ec-189b34607cdc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:27.455387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8116d9c-8fc8-4489-b6ec-189b34607cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:27.455645 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8116d9c-8fc8-4489-b6ec-189b34607cdc None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:27.456355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8116d9c-8fc8-4489-b6ec-189b34607cdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:27.456659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8116d9c-8fc8-4489-b6ec-189b34607cdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:27.458373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8116d9c-8fc8-4489-b6ec-189b34607cdc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:27.472583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 606/1204] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:27.479370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc73d17-e5f6-4ca0-b1ab-abf781db0a3c None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:27.552901 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:27.626563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc73d17-e5f6-4ca0-b1ab-abf781db0a3c None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:27.667213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49206681-6301-44fc-87d8-05b4043c7631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:27.668167 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49206681-6301-44fc-87d8-05b4043c7631 None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:27.668570 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49206681-6301-44fc-87d8-05b4043c7631 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:27.668820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49206681-6301-44fc-87d8-05b4043c7631 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:27.669472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49206681-6301-44fc-87d8-05b4043c7631 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:27.670962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49206681-6301-44fc-87d8-05b4043c7631 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:27.697399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 599/1205] 10.222.0.62 () {60 vars in 1132 bytes} [Thu Sep 1 16:26:27 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:26:29.790704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f82c8-de2f-4fad-af04-3b4b2d79bb85 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:29.791244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f82c8-de2f-4fad-af04-3b4b2d79bb85 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:29.791244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f82c8-de2f-4fad-af04-3b4b2d79bb85 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:29.792925 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 607/1206] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:29.798498 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806f836-8cce-4141-ab87-b58edf986c9f None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:29.799026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806f836-8cce-4141-ab87-b58edf986c9f None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:29.799142 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806f836-8cce-4141-ab87-b58edf986c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:29.821418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-4806f836-8cce-4141-ab87-b58edf986c9f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:29.821418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-4806f836-8cce-4141-ab87-b58edf986c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:29.892623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 600/1207] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:26:30.171071 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f82c8-de2f-4fad-af04-3b4b2d79bb85 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:30.297213 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f82c8-de2f-4fad-af04-3b4b2d79bb85 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:30.343232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0380d-5325-4ca1-b468-b972bb08bbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:30.345163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0380d-5325-4ca1-b468-b972bb08bbe5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:30.345163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0380d-5325-4ca1-b468-b972bb08bbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:30.345163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0380d-5325-4ca1-b468-b972bb08bbe5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:30.359680 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0380d-5325-4ca1-b468-b972bb08bbe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e1f0380d-5325-4ca1-b468-b972bb08bbe5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:26:30.369196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:30.370200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:30.371528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:30.371528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:30.371528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:30.371528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:30.371528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:26:30.371528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:26:30.371528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:26:30.371528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:26:30.371528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:26:30.371528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:26:30.371528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:26:30.371528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:30.371528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 608/1208] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:26:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:30.377904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806f836-8cce-4141-ab87-b58edf986c9f None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:30.423801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:30.506204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806f836-8cce-4141-ab87-b58edf986c9f None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:30.548045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79344f-3c8a-4770-a3ad-8af44f021744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:30.548752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79344f-3c8a-4770-a3ad-8af44f021744 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:30.548946 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79344f-3c8a-4770-a3ad-8af44f021744 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:30.549219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79344f-3c8a-4770-a3ad-8af44f021744 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:30.550034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79344f-3c8a-4770-a3ad-8af44f021744 None admin] RBAC: Adding query filter params (name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:30.550390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79344f-3c8a-4770-a3ad-8af44f021744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:30.551984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79344f-3c8a-4770-a3ad-8af44f021744 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:30.567002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 601/1209] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:26:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:30.568388 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0380d-5325-4ca1-b468-b972bb08bbe5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:30.632077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:30.689239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0380d-5325-4ca1-b468-b972bb08bbe5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:30.728793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da79125-7d06-46c6-9ade-ef87fa2b4509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:30.729720 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da79125-7d06-46c6-9ade-ef87fa2b4509 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:30.730240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da79125-7d06-46c6-9ade-ef87fa2b4509 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:30.730627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da79125-7d06-46c6-9ade-ef87fa2b4509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:30.743174 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da79125-7d06-46c6-9ade-ef87fa2b4509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:30.745058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da79125-7d06-46c6-9ade-ef87fa2b4509 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-0da79125-7d06-46c6-9ade-ef87fa2b4509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:26:30.762200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:30.763473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:26:30.765759 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:26:30.765759 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:30.765759 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:30.765759 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 609/1210] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:26:30.770321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79344f-3c8a-4770-a3ad-8af44f021744 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:30.884953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:30.888546 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79344f-3c8a-4770-a3ad-8af44f021744 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:30.929849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398b884-8de3-4ada-97a1-a8bf358d1877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:30.930476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398b884-8de3-4ada-97a1-a8bf358d1877 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:30.930782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398b884-8de3-4ada-97a1-a8bf358d1877 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:30.931058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398b884-8de3-4ada-97a1-a8bf358d1877 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:30.931664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398b884-8de3-4ada-97a1-a8bf358d1877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:30.931996 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398b884-8de3-4ada-97a1-a8bf358d1877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:30.933611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398b884-8de3-4ada-97a1-a8bf358d1877 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:30.947991 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 602/1211] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:30.952317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da79125-7d06-46c6-9ade-ef87fa2b4509 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:31.059471 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da79125-7d06-46c6-9ade-ef87fa2b4509 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:31.061115 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:31.099225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044510b4-255b-41ff-844c-7b907d5c33e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:31.100063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044510b4-255b-41ff-844c-7b907d5c33e6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:31.100411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044510b4-255b-41ff-844c-7b907d5c33e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:31.100711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044510b4-255b-41ff-844c-7b907d5c33e6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:31.109093 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044510b4-255b-41ff-844c-7b907d5c33e6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:31.110550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044510b4-255b-41ff-844c-7b907d5c33e6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-044510b4-255b-41ff-844c-7b907d5c33e6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:26:31.122846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:31.125330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:31.126355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:31.126355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:31.126355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:26:31.126355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:26:31.126355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:26:31.126355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:26:31.126355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:26:31.126355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:26:31.126355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:26:31.126355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:31.126355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 610/1212] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:31.130833 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398b884-8de3-4ada-97a1-a8bf358d1877 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:31.254845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398b884-8de3-4ada-97a1-a8bf358d1877 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:31.304963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10f69c-91ab-4087-a14e-31c763037d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:31.305598 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10f69c-91ab-4087-a14e-31c763037d12 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:31.305860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10f69c-91ab-4087-a14e-31c763037d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:31.306145 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10f69c-91ab-4087-a14e-31c763037d12 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:31.307201 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10f69c-91ab-4087-a14e-31c763037d12 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:31.307498 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10f69c-91ab-4087-a14e-31c763037d12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:31.309060 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10f69c-91ab-4087-a14e-31c763037d12 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:31.328899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 603/1213] 10.222.0.62 () {56 vars in 1143 bytes} [Thu Sep 1 16:26:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:31.336235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044510b4-255b-41ff-844c-7b907d5c33e6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:31.450776 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044510b4-255b-41ff-844c-7b907d5c33e6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:31.494308 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35143ae-0e68-4b4a-a3c6-a13231da35a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:31.495117 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35143ae-0e68-4b4a-a3c6-a13231da35a8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:31.495494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35143ae-0e68-4b4a-a3c6-a13231da35a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:31.495802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35143ae-0e68-4b4a-a3c6-a13231da35a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:31.535802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:31.589667 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35143ae-0e68-4b4a-a3c6-a13231da35a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:31.591128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35143ae-0e68-4b4a-a3c6-a13231da35a8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:31.715956 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 611/1214] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:26:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:26:31.722500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10f69c-91ab-4087-a14e-31c763037d12 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:31.753868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:31.848020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10f69c-91ab-4087-a14e-31c763037d12 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:31.892715 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3379cf-f88a-4541-a032-92ae8ecfbcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:31.896543 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3379cf-f88a-4541-a032-92ae8ecfbcd0 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:31.896543 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3379cf-f88a-4541-a032-92ae8ecfbcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:31.896543 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3379cf-f88a-4541-a032-92ae8ecfbcd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:31.906101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3379cf-f88a-4541-a032-92ae8ecfbcd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:31.906581 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3379cf-f88a-4541-a032-92ae8ecfbcd0 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-8c3379cf-f88a-4541-a032-92ae8ecfbcd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:26:31.923010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:31.924069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:26:31.924963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:26:31.924963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:31.924963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:31.924963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 604/1215] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:31.930263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35143ae-0e68-4b4a-a3c6-a13231da35a8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:31.994352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:32.040370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35143ae-0e68-4b4a-a3c6-a13231da35a8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:32.076460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0486c-6a1b-4bc7-8cce-61a26c5d5f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:32.077069 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0486c-6a1b-4bc7-8cce-61a26c5d5f60 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:32.077343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0486c-6a1b-4bc7-8cce-61a26c5d5f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:32.077607 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0486c-6a1b-4bc7-8cce-61a26c5d5f60 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:32.078266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0486c-6a1b-4bc7-8cce-61a26c5d5f60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:32.078601 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0486c-6a1b-4bc7-8cce-61a26c5d5f60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:32.080816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0486c-6a1b-4bc7-8cce-61a26c5d5f60 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:32.097316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 612/1216] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:32.099063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3379cf-f88a-4541-a032-92ae8ecfbcd0 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:32.183204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:32.224569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3379cf-f88a-4541-a032-92ae8ecfbcd0 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:32.262318 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83b6f0-cf16-4bbd-9664-c4ad4db03f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:32.263039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83b6f0-cf16-4bbd-9664-c4ad4db03f47 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:32.263371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83b6f0-cf16-4bbd-9664-c4ad4db03f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:32.263698 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83b6f0-cf16-4bbd-9664-c4ad4db03f47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:32.271393 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83b6f0-cf16-4bbd-9664-c4ad4db03f47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:32.273075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83b6f0-cf16-4bbd-9664-c4ad4db03f47 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-df83b6f0-cf16-4bbd-9664-c4ad4db03f47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:26:32.286806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:32.288417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:32.289451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:26:32.289451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:26:32.289451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:26:32.289451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:26:32.289451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:26:32.289451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:26:32.289451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:26:32.289451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:32.290007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 605/1217] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:26:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:26:32.301377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0486c-6a1b-4bc7-8cce-61a26c5d5f60 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:32.422810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0486c-6a1b-4bc7-8cce-61a26c5d5f60 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:32.450056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:32.462953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb833da2-4e52-4b52-b2aa-9a248e0c5ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:32.463568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb833da2-4e52-4b52-b2aa-9a248e0c5ecf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:32.463847 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb833da2-4e52-4b52-b2aa-9a248e0c5ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:32.464179 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb833da2-4e52-4b52-b2aa-9a248e0c5ecf None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:32.465043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb833da2-4e52-4b52-b2aa-9a248e0c5ecf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:32.465342 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb833da2-4e52-4b52-b2aa-9a248e0c5ecf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:32.466768 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb833da2-4e52-4b52-b2aa-9a248e0c5ecf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:32.481553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 613/1218] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:26:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:26:32.487872 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83b6f0-cf16-4bbd-9664-c4ad4db03f47 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:32.607180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83b6f0-cf16-4bbd-9664-c4ad4db03f47 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:32.642811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:32.652274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea2320-cf7d-45a1-84b0-30a147e10291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:32.653149 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea2320-cf7d-45a1-84b0-30a147e10291 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:32.653489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea2320-cf7d-45a1-84b0-30a147e10291 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:32.653824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea2320-cf7d-45a1-84b0-30a147e10291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:32.654626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea2320-cf7d-45a1-84b0-30a147e10291 None admin] RBAC: Adding query filter params (role.id=fc3db8095f31464ca4c378645d1d078b, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=6281dcf2927a4ed2870125df8801be78) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:32.655125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea2320-cf7d-45a1-84b0-30a147e10291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:32.656698 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea2320-cf7d-45a1-84b0-30a147e10291 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:32.671934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 606/1219] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:26:32 2022] GET /identity/v3/role_assignments?user.id=6281dcf2927a4ed2870125df8801be78&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=fc3db8095f31464ca4c378645d1d078b => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:34.688265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc697b8-2589-4500-9387-e0af3b7a0ce7 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:34.688849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc697b8-2589-4500-9387-e0af3b7a0ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:34.688849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc697b8-2589-4500-9387-e0af3b7a0ce7 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:34.690199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 614/1220] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:34.695933 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7aca4-b2e3-4de8-94fe-84157d378602 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:34.696461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7aca4-b2e3-4de8-94fe-84157d378602 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:34.696461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7aca4-b2e3-4de8-94fe-84157d378602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:34.716174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-73d7aca4-b2e3-4de8-94fe-84157d378602 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:34.717382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-73d7aca4-b2e3-4de8-94fe-84157d378602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:34.792666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 607/1221] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:26:35.078529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc697b8-2589-4500-9387-e0af3b7a0ce7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:35.193336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc697b8-2589-4500-9387-e0af3b7a0ce7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:35.233478 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16885568-a2c1-4701-b225-d56991a5be1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:35.234192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16885568-a2c1-4701-b225-d56991a5be1c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:35.234386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16885568-a2c1-4701-b225-d56991a5be1c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:35.234657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16885568-a2c1-4701-b225-d56991a5be1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:35.245060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16885568-a2c1-4701-b225-d56991a5be1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-16885568-a2c1-4701-b225-d56991a5be1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:26:35.249887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:35.251078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:35.252137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:35.252137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:35.252137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:35.252137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:35.252137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:26:35.252137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:26:35.252137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:26:35.252137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:26:35.252137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:26:35.252137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:26:35.252137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:26:35.252137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:35.252137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 615/1222] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:26:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:35.257747 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7aca4-b2e3-4de8-94fe-84157d378602 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:35.362614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7aca4-b2e3-4de8-94fe-84157d378602 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:35.386455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:35.398963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e45b4-f220-4720-b6e9-43e637a99c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:35.399577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e45b4-f220-4720-b6e9-43e637a99c53 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:35.399909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e45b4-f220-4720-b6e9-43e637a99c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:35.400205 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e45b4-f220-4720-b6e9-43e637a99c53 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:35.400967 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e45b4-f220-4720-b6e9-43e637a99c53 None admin] RBAC: Adding query filter params (name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:35.401334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e45b4-f220-4720-b6e9-43e637a99c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:35.402897 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e45b4-f220-4720-b6e9-43e637a99c53 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:35.416927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 608/1223] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:26:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:35.418340 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16885568-a2c1-4701-b225-d56991a5be1c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:35.466322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:35.523418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16885568-a2c1-4701-b225-d56991a5be1c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:35.556622 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98fd4b-ad02-40cc-b8a8-40a999ba7509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:35.557242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98fd4b-ad02-40cc-b8a8-40a999ba7509 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:35.557512 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98fd4b-ad02-40cc-b8a8-40a999ba7509 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:35.557801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98fd4b-ad02-40cc-b8a8-40a999ba7509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:35.567562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98fd4b-ad02-40cc-b8a8-40a999ba7509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:35.569023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98fd4b-ad02-40cc-b8a8-40a999ba7509 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-df98fd4b-ad02-40cc-b8a8-40a999ba7509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:26:35.584545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:35.585577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:26:35.586472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:26:35.586472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:35.586472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:35.586472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 616/1224] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:26:35.592734 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e45b4-f220-4720-b6e9-43e637a99c53 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:35.698346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e45b4-f220-4720-b6e9-43e637a99c53 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:35.733377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efd375-f511-4dc6-9737-6b911fee7346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:35.733983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efd375-f511-4dc6-9737-6b911fee7346 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:35.734258 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efd375-f511-4dc6-9737-6b911fee7346 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:35.734522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efd375-f511-4dc6-9737-6b911fee7346 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:35.735158 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efd375-f511-4dc6-9737-6b911fee7346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:35.735451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efd375-f511-4dc6-9737-6b911fee7346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:35.736845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efd375-f511-4dc6-9737-6b911fee7346 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:35.748405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 609/1225] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:35.754725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98fd4b-ad02-40cc-b8a8-40a999ba7509 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:35.859096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:35.867170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98fd4b-ad02-40cc-b8a8-40a999ba7509 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:35.901030 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bfa9d-0241-48e9-b33b-a7f0256d550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:35.901796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bfa9d-0241-48e9-b33b-a7f0256d550e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:35.902074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bfa9d-0241-48e9-b33b-a7f0256d550e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:35.902407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bfa9d-0241-48e9-b33b-a7f0256d550e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:35.909980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bfa9d-0241-48e9-b33b-a7f0256d550e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:35.911430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bfa9d-0241-48e9-b33b-a7f0256d550e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b77bfa9d-0241-48e9-b33b-a7f0256d550e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:26:35.922634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:35.923782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:35.924923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:35.924923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:35.924923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:26:35.924923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:26:35.924923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:26:35.924923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:26:35.924923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:26:35.924923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:26:35.924923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:26:35.924923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:35.924923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 617/1226] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:35.929838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efd375-f511-4dc6-9737-6b911fee7346 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:35.978251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:36.035366 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efd375-f511-4dc6-9737-6b911fee7346 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:36.075690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84654648-fef5-4fed-91ac-376fecf51df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:36.076332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84654648-fef5-4fed-91ac-376fecf51df1 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:36.076607 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84654648-fef5-4fed-91ac-376fecf51df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:36.076871 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84654648-fef5-4fed-91ac-376fecf51df1 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:36.077607 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84654648-fef5-4fed-91ac-376fecf51df1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:36.077906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84654648-fef5-4fed-91ac-376fecf51df1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:36.079342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84654648-fef5-4fed-91ac-376fecf51df1 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:36.097910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 610/1227] 10.222.0.62 () {56 vars in 1143 bytes} [Thu Sep 1 16:26:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:36.103735 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bfa9d-0241-48e9-b33b-a7f0256d550e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:36.210346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bfa9d-0241-48e9-b33b-a7f0256d550e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:36.246739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46985fd8-9fa3-4b44-8dea-f2984b7e5c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:36.247335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46985fd8-9fa3-4b44-8dea-f2984b7e5c7a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:36.247610 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46985fd8-9fa3-4b44-8dea-f2984b7e5c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:36.247881 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46985fd8-9fa3-4b44-8dea-f2984b7e5c7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:36.259614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46985fd8-9fa3-4b44-8dea-f2984b7e5c7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:36.261106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46985fd8-9fa3-4b44-8dea-f2984b7e5c7a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-46985fd8-9fa3-4b44-8dea-f2984b7e5c7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:26:36.277521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:36.278810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:26:36.279811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:26:36.279811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:36.279811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:36.279811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 618/1228] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:36.285794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84654648-fef5-4fed-91ac-376fecf51df1 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:36.398851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84654648-fef5-4fed-91ac-376fecf51df1 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:36.401436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:36.437184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e91d16-c880-4dbf-8927-973b13926769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:36.437841 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e91d16-c880-4dbf-8927-973b13926769 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:36.438204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e91d16-c880-4dbf-8927-973b13926769 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:36.438436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e91d16-c880-4dbf-8927-973b13926769 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:36.439107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e91d16-c880-4dbf-8927-973b13926769 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:36.439438 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e91d16-c880-4dbf-8927-973b13926769 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:36.440917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e91d16-c880-4dbf-8927-973b13926769 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:36.454279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 611/1229] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:36.462170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46985fd8-9fa3-4b44-8dea-f2984b7e5c7a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:36.475287 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:36.576424 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46985fd8-9fa3-4b44-8dea-f2984b7e5c7a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:36.618487 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eafcf97-ee2a-42fe-a45e-0fd45ac30654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:36.619161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafcf97-ee2a-42fe-a45e-0fd45ac30654 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:36.619490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafcf97-ee2a-42fe-a45e-0fd45ac30654 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:36.619763 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafcf97-ee2a-42fe-a45e-0fd45ac30654 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:36.627047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eafcf97-ee2a-42fe-a45e-0fd45ac30654 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:36.628542 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eafcf97-ee2a-42fe-a45e-0fd45ac30654 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2eafcf97-ee2a-42fe-a45e-0fd45ac30654 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:26:36.641153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:36.642282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:36.643443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:26:36.643443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:26:36.643443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:26:36.643443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:26:36.643443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:26:36.643443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:26:36.643443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:26:36.643443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:36.644413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 619/1230] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:26:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:36.650419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e91d16-c880-4dbf-8927-973b13926769 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:36.753812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e91d16-c880-4dbf-8927-973b13926769 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:36.791504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda7995-7971-4e94-9cc5-6e2e4ff86e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:36.792382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda7995-7971-4e94-9cc5-6e2e4ff86e70 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:36.792571 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda7995-7971-4e94-9cc5-6e2e4ff86e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:36.792812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda7995-7971-4e94-9cc5-6e2e4ff86e70 None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:36.793540 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda7995-7971-4e94-9cc5-6e2e4ff86e70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:36.793848 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda7995-7971-4e94-9cc5-6e2e4ff86e70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:36.795292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda7995-7971-4e94-9cc5-6e2e4ff86e70 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:36.808295 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 612/1231] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:26:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:26:36.814433 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eafcf97-ee2a-42fe-a45e-0fd45ac30654 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:36.885984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:36.923596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eafcf97-ee2a-42fe-a45e-0fd45ac30654 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:36.959281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7314cea-545b-4492-b267-5846e06d9ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:36.960069 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7314cea-545b-4492-b267-5846e06d9ae6 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:36.960387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7314cea-545b-4492-b267-5846e06d9ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:36.960706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7314cea-545b-4492-b267-5846e06d9ae6 None admin] PATH_INFO: `/v3/projects/425048ffd13448cd8fe815333a9b798c/users/6281dcf2927a4ed2870125df8801be78/roles/fc3db8095f31464ca4c378645d1d078b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:36.976539 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7314cea-545b-4492-b267-5846e06d9ae6 None admin] RBAC: Authorizing `identity:create_grant(project_id=425048ffd13448cd8fe815333a9b798c, user_id=6281dcf2927a4ed2870125df8801be78, role_id=fc3db8095f31464ca4c378645d1d078b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:36.978099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7314cea-545b-4492-b267-5846e06d9ae6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:36.993184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 620/1232] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:26:36 2022] PUT /identity/v3/projects/425048ffd13448cd8fe815333a9b798c/users/6281dcf2927a4ed2870125df8801be78/roles/fc3db8095f31464ca4c378645d1d078b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:26:38.933195 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e66ddf-a647-40bd-8735-b7e744034f58 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:38.933844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e66ddf-a647-40bd-8735-b7e744034f58 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:38.933844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e66ddf-a647-40bd-8735-b7e744034f58 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:38.935043 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 613/1233] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:38.942246 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51349801-806e-4a30-ba3e-81a1a21ee35f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:38.942680 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51349801-806e-4a30-ba3e-81a1a21ee35f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:38.942786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51349801-806e-4a30-ba3e-81a1a21ee35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:38.962563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-51349801-806e-4a30-ba3e-81a1a21ee35f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:38.963760 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-51349801-806e-4a30-ba3e-81a1a21ee35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:39.037509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 621/1234] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:26:39.312862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e66ddf-a647-40bd-8735-b7e744034f58 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:39.340677 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:39.419803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e66ddf-a647-40bd-8735-b7e744034f58 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:39.456489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cef35a-ce59-452a-aaea-be0f9a4210f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:39.457493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cef35a-ce59-452a-aaea-be0f9a4210f5 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:39.457803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cef35a-ce59-452a-aaea-be0f9a4210f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:39.458131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cef35a-ce59-452a-aaea-be0f9a4210f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:39.469722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cef35a-ce59-452a-aaea-be0f9a4210f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-06cef35a-ce59-452a-aaea-be0f9a4210f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:26:39.474675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:39.475653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:39.476692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:39.476692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:39.476692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:39.476692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:39.476692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:26:39.476692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:26:39.476692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:26:39.476692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:26:39.476692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:26:39.476692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:26:39.476692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:26:39.476692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:39.476692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 614/1235] 10.222.0.62 () {56 vars in 1102 bytes} [Thu Sep 1 16:26:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:39.482225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51349801-806e-4a30-ba3e-81a1a21ee35f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:39.530575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:39.590873 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51349801-806e-4a30-ba3e-81a1a21ee35f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:39.627631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606f3bc-8aff-4c0a-b6e4-329a7c318c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:39.628266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606f3bc-8aff-4c0a-b6e4-329a7c318c3f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:39.628556 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606f3bc-8aff-4c0a-b6e4-329a7c318c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:39.628827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606f3bc-8aff-4c0a-b6e4-329a7c318c3f None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:39.629488 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606f3bc-8aff-4c0a-b6e4-329a7c318c3f None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:39.629792 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606f3bc-8aff-4c0a-b6e4-329a7c318c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:39.631168 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606f3bc-8aff-4c0a-b6e4-329a7c318c3f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:39.639667 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 622/1236] 10.222.0.62 () {56 vars in 1103 bytes} [Thu Sep 1 16:26:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:39.645722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cef35a-ce59-452a-aaea-be0f9a4210f5 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:39.737977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:39.758748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cef35a-ce59-452a-aaea-be0f9a4210f5 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:39.802632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fedff-7b60-47fb-81bb-6cf382a1ca7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:39.803283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fedff-7b60-47fb-81bb-6cf382a1ca7f None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:39.803562 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fedff-7b60-47fb-81bb-6cf382a1ca7f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:39.803833 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fedff-7b60-47fb-81bb-6cf382a1ca7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:39.816282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114fedff-7b60-47fb-81bb-6cf382a1ca7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:39.818044 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114fedff-7b60-47fb-81bb-6cf382a1ca7f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-114fedff-7b60-47fb-81bb-6cf382a1ca7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:26:39.835315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:39.836436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:26:39.837405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:26:39.837405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:39.837405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:39.842322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 615/1237] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:39.844841 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606f3bc-8aff-4c0a-b6e4-329a7c318c3f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:39.963565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606f3bc-8aff-4c0a-b6e4-329a7c318c3f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:39.973016 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:40.004101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4f97a-2d70-4fd6-9381-e72b4c2d8204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:40.004791 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4f97a-2d70-4fd6-9381-e72b4c2d8204 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:40.005178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4f97a-2d70-4fd6-9381-e72b4c2d8204 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:40.005587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4f97a-2d70-4fd6-9381-e72b4c2d8204 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:40.006288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4f97a-2d70-4fd6-9381-e72b4c2d8204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:40.006587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4f97a-2d70-4fd6-9381-e72b4c2d8204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:40.008034 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4f97a-2d70-4fd6-9381-e72b4c2d8204 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:40.021057 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 623/1238] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:40.027652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fedff-7b60-47fb-81bb-6cf382a1ca7f None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:40.137219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fedff-7b60-47fb-81bb-6cf382a1ca7f None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:40.174050 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ba30f-647c-4d54-919f-b3c8b383c7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:40.174692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ba30f-647c-4d54-919f-b3c8b383c7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:40.174969 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ba30f-647c-4d54-919f-b3c8b383c7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:40.175240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ba30f-647c-4d54-919f-b3c8b383c7a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:40.178140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:40.185482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ba30f-647c-4d54-919f-b3c8b383c7a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:40.187207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ba30f-647c-4d54-919f-b3c8b383c7a8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-0b4ba30f-647c-4d54-919f-b3c8b383c7a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:26:40.200631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:40.201786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:40.202977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:40.202977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:40.202977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:26:40.202977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:26:40.202977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:26:40.202977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:26:40.202977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:26:40.202977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:26:40.202977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:26:40.202977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:40.202977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 616/1239] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:40.210221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4f97a-2d70-4fd6-9381-e72b4c2d8204 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:40.317179 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4f97a-2d70-4fd6-9381-e72b4c2d8204 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:40.360111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec89d5-d973-4d69-a968-6f38f2959ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:40.360725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec89d5-d973-4d69-a968-6f38f2959ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:40.360997 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec89d5-d973-4d69-a968-6f38f2959ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:40.361261 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec89d5-d973-4d69-a968-6f38f2959ac6 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:40.362005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec89d5-d973-4d69-a968-6f38f2959ac6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:40.362287 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec89d5-d973-4d69-a968-6f38f2959ac6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:40.363713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec89d5-d973-4d69-a968-6f38f2959ac6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:40.381656 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 624/1240] 10.222.0.62 () {56 vars in 1143 bytes} [Thu Sep 1 16:26:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:40.389579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ba30f-647c-4d54-919f-b3c8b383c7a8 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:40.500723 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ba30f-647c-4d54-919f-b3c8b383c7a8 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:40.536632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9ed6f-a47f-4cf9-a583-cb69c789ee06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:40.537259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9ed6f-a47f-4cf9-a583-cb69c789ee06 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:40.537536 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9ed6f-a47f-4cf9-a583-cb69c789ee06 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:40.537807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9ed6f-a47f-4cf9-a583-cb69c789ee06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:40.607235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:40.614093 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9ed6f-a47f-4cf9-a583-cb69c789ee06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:40.615562 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9ed6f-a47f-4cf9-a583-cb69c789ee06 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:40.720212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 617/1241] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:26:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:26:40.726243 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec89d5-d973-4d69-a968-6f38f2959ac6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:40.758223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:40.838019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec89d5-d973-4d69-a968-6f38f2959ac6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:40.874525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186123d4-1de4-4e64-90ed-5c08282eb1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:40.875157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186123d4-1de4-4e64-90ed-5c08282eb1ba None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:40.875430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186123d4-1de4-4e64-90ed-5c08282eb1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:40.875692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186123d4-1de4-4e64-90ed-5c08282eb1ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:40.887765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186123d4-1de4-4e64-90ed-5c08282eb1ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:40.889280 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186123d4-1de4-4e64-90ed-5c08282eb1ba None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-186123d4-1de4-4e64-90ed-5c08282eb1ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:26:40.905941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:40.907184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:26:40.908345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:26:40.908345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:40.908345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:40.908345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 625/1242] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:40.913367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9ed6f-a47f-4cf9-a583-cb69c789ee06 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:41.015266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:41.021979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9ed6f-a47f-4cf9-a583-cb69c789ee06 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:41.058554 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420dc29-a43f-4d47-86bd-8cc55c815292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:41.059187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420dc29-a43f-4d47-86bd-8cc55c815292 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:41.059421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420dc29-a43f-4d47-86bd-8cc55c815292 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:41.059689 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420dc29-a43f-4d47-86bd-8cc55c815292 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:41.060459 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420dc29-a43f-4d47-86bd-8cc55c815292 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:41.060758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420dc29-a43f-4d47-86bd-8cc55c815292 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:41.062139 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420dc29-a43f-4d47-86bd-8cc55c815292 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:41.073653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 618/1243] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:41.079390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186123d4-1de4-4e64-90ed-5c08282eb1ba None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:41.164440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:41.187282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186123d4-1de4-4e64-90ed-5c08282eb1ba None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:41.222767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88af70-5f40-4ae3-adc0-3eb72af38277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:41.223457 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88af70-5f40-4ae3-adc0-3eb72af38277 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:41.223799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88af70-5f40-4ae3-adc0-3eb72af38277 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:41.224061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88af70-5f40-4ae3-adc0-3eb72af38277 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:41.231640 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88af70-5f40-4ae3-adc0-3eb72af38277 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:41.233116 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88af70-5f40-4ae3-adc0-3eb72af38277 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-5b88af70-5f40-4ae3-adc0-3eb72af38277 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:26:41.245544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:41.246503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:41.247382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:26:41.247382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:26:41.247382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:26:41.247382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:26:41.247382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:26:41.247382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:26:41.247382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:26:41.247382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:41.247382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 626/1244] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:26:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:41.252962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420dc29-a43f-4d47-86bd-8cc55c815292 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:41.372430 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420dc29-a43f-4d47-86bd-8cc55c815292 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:41.410483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2917bbf-440f-44d2-b26b-30943f1281bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:41.411113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2917bbf-440f-44d2-b26b-30943f1281bc None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:41.411388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2917bbf-440f-44d2-b26b-30943f1281bc None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:41.411655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2917bbf-440f-44d2-b26b-30943f1281bc None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:41.412414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2917bbf-440f-44d2-b26b-30943f1281bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:41.412678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2917bbf-440f-44d2-b26b-30943f1281bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:41.414131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2917bbf-440f-44d2-b26b-30943f1281bc None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:41.427717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 619/1245] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:26:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:41.434301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88af70-5f40-4ae3-adc0-3eb72af38277 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:41.546631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88af70-5f40-4ae3-adc0-3eb72af38277 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:41.580621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78811ff5-9269-4f3f-abb0-9e36d62d7e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:41.581290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78811ff5-9269-4f3f-abb0-9e36d62d7e60 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:41.581892 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78811ff5-9269-4f3f-abb0-9e36d62d7e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:41.582188 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78811ff5-9269-4f3f-abb0-9e36d62d7e60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:41.583003 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78811ff5-9269-4f3f-abb0-9e36d62d7e60 None admin] RBAC: Adding query filter params (role.id=fc3db8095f31464ca4c378645d1d078b, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=6281dcf2927a4ed2870125df8801be78) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:41.583354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78811ff5-9269-4f3f-abb0-9e36d62d7e60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:41.584881 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78811ff5-9269-4f3f-abb0-9e36d62d7e60 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:41.591821 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 627/1246] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:26:41 2022] GET /identity/v3/role_assignments?user.id=6281dcf2927a4ed2870125df8801be78&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=fc3db8095f31464ca4c378645d1d078b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:43.759979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7893d2-d4e3-415d-b118-ee924beb98fe None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:43.760554 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7893d2-d4e3-415d-b118-ee924beb98fe None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:43.760554 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7893d2-d4e3-415d-b118-ee924beb98fe None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:43.761898 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 620/1247] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:43.768495 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1b169-e7ec-4bc1-b293-37b4f9130e1d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:43.769030 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1b169-e7ec-4bc1-b293-37b4f9130e1d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:43.769030 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1b169-e7ec-4bc1-b293-37b4f9130e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:43.776495 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:43.793020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d0a1b169-e7ec-4bc1-b293-37b4f9130e1d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:43.794566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d0a1b169-e7ec-4bc1-b293-37b4f9130e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:43.882402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 628/1248] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:26:44.149590 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7893d2-d4e3-415d-b118-ee924beb98fe None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:44.170670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:44.266851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7893d2-d4e3-415d-b118-ee924beb98fe None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:44.311155 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd126c6-b518-40e1-9717-963826dd3371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:44.311894 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd126c6-b518-40e1-9717-963826dd3371 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:44.312196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd126c6-b518-40e1-9717-963826dd3371 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:44.312656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd126c6-b518-40e1-9717-963826dd3371 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:44.324499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd126c6-b518-40e1-9717-963826dd3371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-3bd126c6-b518-40e1-9717-963826dd3371 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:26:44.329620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:44.331382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:44.332913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:44.332913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:44.332913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:44.332913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:44.332913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:26:44.332913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:26:44.332913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:26:44.332913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:26:44.332913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:26:44.332913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:26:44.332913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:26:44.332913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:44.332913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 621/1249] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:26:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:44.338167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1b169-e7ec-4bc1-b293-37b4f9130e1d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:44.449108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1b169-e7ec-4bc1-b293-37b4f9130e1d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:44.481196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:44.487156 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0e41e-1a62-4d78-b7dd-954ba68f4b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:44.487921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0e41e-1a62-4d78-b7dd-954ba68f4b60 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:44.488213 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0e41e-1a62-4d78-b7dd-954ba68f4b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:44.488483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0e41e-1a62-4d78-b7dd-954ba68f4b60 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:44.489210 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0e41e-1a62-4d78-b7dd-954ba68f4b60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:44.489430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0e41e-1a62-4d78-b7dd-954ba68f4b60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:44.490977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0e41e-1a62-4d78-b7dd-954ba68f4b60 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:44.501165 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 629/1250] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:26:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:44.507316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd126c6-b518-40e1-9717-963826dd3371 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:44.598108 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:44.624434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd126c6-b518-40e1-9717-963826dd3371 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:44.662530 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0002a1-9d2f-438f-84b9-c754ff25ed8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:44.663167 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0002a1-9d2f-438f-84b9-c754ff25ed8f None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:44.663442 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0002a1-9d2f-438f-84b9-c754ff25ed8f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:44.663706 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0002a1-9d2f-438f-84b9-c754ff25ed8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:44.675901 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0002a1-9d2f-438f-84b9-c754ff25ed8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:44.677430 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0002a1-9d2f-438f-84b9-c754ff25ed8f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-ed0002a1-9d2f-438f-84b9-c754ff25ed8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:26:44.693597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:44.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:26:44.695962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:26:44.695962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:44.695962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:44.695962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 622/1251] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:44.702699 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0e41e-1a62-4d78-b7dd-954ba68f4b60 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:44.818100 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0e41e-1a62-4d78-b7dd-954ba68f4b60 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:44.855201 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d77d2c-f2f5-4431-8ad8-7bfe2d1e6a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:44.855816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d77d2c-f2f5-4431-8ad8-7bfe2d1e6a97 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:44.856126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d77d2c-f2f5-4431-8ad8-7bfe2d1e6a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:44.856397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d77d2c-f2f5-4431-8ad8-7bfe2d1e6a97 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:44.858020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d77d2c-f2f5-4431-8ad8-7bfe2d1e6a97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:44.858335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d77d2c-f2f5-4431-8ad8-7bfe2d1e6a97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:44.859871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d77d2c-f2f5-4431-8ad8-7bfe2d1e6a97 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:44.873562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 630/1252] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:44.879177 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0002a1-9d2f-438f-84b9-c754ff25ed8f None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:44.988098 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0002a1-9d2f-438f-84b9-c754ff25ed8f None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:45.017663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:45.025808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26f963-a3ee-4ccd-9ac2-f95f02acb59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:45.026456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26f963-a3ee-4ccd-9ac2-f95f02acb59a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:45.026729 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26f963-a3ee-4ccd-9ac2-f95f02acb59a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:45.026996 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26f963-a3ee-4ccd-9ac2-f95f02acb59a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:45.036065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26f963-a3ee-4ccd-9ac2-f95f02acb59a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:45.037538 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26f963-a3ee-4ccd-9ac2-f95f02acb59a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-af26f963-a3ee-4ccd-9ac2-f95f02acb59a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:26:45.049910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:45.051123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:45.052510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:45.052510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:45.052510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:26:45.052510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:26:45.052510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:26:45.052510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:26:45.052510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:26:45.052510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:26:45.052510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:26:45.052510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:45.052510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 623/1253] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:26:45.057589 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d77d2c-f2f5-4431-8ad8-7bfe2d1e6a97 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:45.124691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:45.180162 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d77d2c-f2f5-4431-8ad8-7bfe2d1e6a97 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:45.221318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84393c-01c2-4fa8-9da2-f926001f7649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:45.222015 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84393c-01c2-4fa8-9da2-f926001f7649 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:45.222355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84393c-01c2-4fa8-9da2-f926001f7649 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:45.222687 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84393c-01c2-4fa8-9da2-f926001f7649 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:45.223510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84393c-01c2-4fa8-9da2-f926001f7649 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:45.223875 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84393c-01c2-4fa8-9da2-f926001f7649 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:45.225540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84393c-01c2-4fa8-9da2-f926001f7649 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:45.247425 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 631/1254] 10.222.0.62 () {56 vars in 1143 bytes} [Thu Sep 1 16:26:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:26:45.253556 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26f963-a3ee-4ccd-9ac2-f95f02acb59a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:45.369376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26f963-a3ee-4ccd-9ac2-f95f02acb59a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:45.407109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0904ed-aa1f-40dc-9656-d81ad1ce5406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:45.407710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0904ed-aa1f-40dc-9656-d81ad1ce5406 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:45.408054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0904ed-aa1f-40dc-9656-d81ad1ce5406 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:45.408321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0904ed-aa1f-40dc-9656-d81ad1ce5406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:45.465459 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:45.477998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0904ed-aa1f-40dc-9656-d81ad1ce5406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:45.479665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0904ed-aa1f-40dc-9656-d81ad1ce5406 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:45.584025 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 624/1255] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:26:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:26:45.590617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84393c-01c2-4fa8-9da2-f926001f7649 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:45.703558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84393c-01c2-4fa8-9da2-f926001f7649 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:45.718414 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:45.740539 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba12bd-c589-4d00-9b62-491a21d20932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:45.741173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba12bd-c589-4d00-9b62-491a21d20932 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:45.741445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba12bd-c589-4d00-9b62-491a21d20932 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:45.741706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba12bd-c589-4d00-9b62-491a21d20932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:45.752044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba12bd-c589-4d00-9b62-491a21d20932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:45.753624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba12bd-c589-4d00-9b62-491a21d20932 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-91ba12bd-c589-4d00-9b62-491a21d20932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:26:45.769393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:45.771073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:26:45.772039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:26:45.772039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:45.772039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:45.772039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 632/1256] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:45.777632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0904ed-aa1f-40dc-9656-d81ad1ce5406 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:45.898845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:45.914362 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0904ed-aa1f-40dc-9656-d81ad1ce5406 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:45.957945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab72b78-8cd3-42ed-bee5-025638b4fd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:45.958559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab72b78-8cd3-42ed-bee5-025638b4fd5a None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:45.958827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab72b78-8cd3-42ed-bee5-025638b4fd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:45.959084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab72b78-8cd3-42ed-bee5-025638b4fd5a None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:45.959717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab72b78-8cd3-42ed-bee5-025638b4fd5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:45.960063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab72b78-8cd3-42ed-bee5-025638b4fd5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:45.961433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab72b78-8cd3-42ed-bee5-025638b4fd5a None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:45.974641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 625/1257] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:45.983179 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba12bd-c589-4d00-9b62-491a21d20932 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:46.100015 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba12bd-c589-4d00-9b62-491a21d20932 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:46.137357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d51fa-7891-409a-964e-ef9a2dfdd9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:46.138203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d51fa-7891-409a-964e-ef9a2dfdd9ba None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:46.138466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d51fa-7891-409a-964e-ef9a2dfdd9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:46.138886 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d51fa-7891-409a-964e-ef9a2dfdd9ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:46.147410 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d51fa-7891-409a-964e-ef9a2dfdd9ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:46.149647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d51fa-7891-409a-964e-ef9a2dfdd9ba None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-106d51fa-7891-409a-964e-ef9a2dfdd9ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:26:46.161148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:46.162441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:46.163533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:26:46.163533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:26:46.163533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:26:46.163533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:26:46.163533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:26:46.163533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:26:46.163533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:26:46.163533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:46.163533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 633/1258] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:26:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:46.168438 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab72b78-8cd3-42ed-bee5-025638b4fd5a None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:46.275484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab72b78-8cd3-42ed-bee5-025638b4fd5a None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:46.318368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f069132-4367-4cad-a30d-357dbd470cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:46.319007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f069132-4367-4cad-a30d-357dbd470cce None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:46.319304 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f069132-4367-4cad-a30d-357dbd470cce None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:46.319571 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f069132-4367-4cad-a30d-357dbd470cce None admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:46.320286 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f069132-4367-4cad-a30d-357dbd470cce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:46.320588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f069132-4367-4cad-a30d-357dbd470cce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:46.322025 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f069132-4367-4cad-a30d-357dbd470cce None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:46.342476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 626/1259] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:26:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:46.343968 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d51fa-7891-409a-964e-ef9a2dfdd9ba None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:46.373039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:46.462393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d51fa-7891-409a-964e-ef9a2dfdd9ba None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:46.500611 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d2995-b204-4cfc-9245-d248341f2e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:46.501398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d2995-b204-4cfc-9245-d248341f2e7b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:46.501690 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d2995-b204-4cfc-9245-d248341f2e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:46.501953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d2995-b204-4cfc-9245-d248341f2e7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:46.502715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d2995-b204-4cfc-9245-d248341f2e7b None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=6281dcf2927a4ed2870125df8801be78) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:46.503142 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d2995-b204-4cfc-9245-d248341f2e7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:46.504789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d2995-b204-4cfc-9245-d248341f2e7b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:46.511637 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 634/1260] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:26:46 2022] GET /identity/v3/role_assignments?user.id=6281dcf2927a4ed2870125df8801be78&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:26:48.401785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56839c54-f3b4-4af0-83f1-8710ff1f8437 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:48.402722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56839c54-f3b4-4af0-83f1-8710ff1f8437 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:48.402722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56839c54-f3b4-4af0-83f1-8710ff1f8437 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:48.404824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 627/1261] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:26:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:48.411070 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5f0d1-a41c-4ad8-951b-312faf2cfeb9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:48.411869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5f0d1-a41c-4ad8-951b-312faf2cfeb9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:48.411869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5f0d1-a41c-4ad8-951b-312faf2cfeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:48.433750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-79e5f0d1-a41c-4ad8-951b-312faf2cfeb9 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:48.434951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-79e5f0d1-a41c-4ad8-951b-312faf2cfeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:48.520168 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 635/1262] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:26:48.784744 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56839c54-f3b4-4af0-83f1-8710ff1f8437 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:48.802735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:48.904108 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56839c54-f3b4-4af0-83f1-8710ff1f8437 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:48.940171 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53ca06-65d0-4437-aa03-c7a8bf476006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:48.940787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53ca06-65d0-4437-aa03-c7a8bf476006 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:48.941061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53ca06-65d0-4437-aa03-c7a8bf476006 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:48.941330 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53ca06-65d0-4437-aa03-c7a8bf476006 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:48.952724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53ca06-65d0-4437-aa03-c7a8bf476006 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-1f53ca06-65d0-4437-aa03-c7a8bf476006 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:26:48.957887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:48.958955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:48.960070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:48.960070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:48.960070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:48.960070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:48.960070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:26:48.960070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:26:48.960070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:26:48.960070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:26:48.960070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:26:48.960070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:26:48.960070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:26:48.960070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:48.960070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 628/1263] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:26:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:48.969468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5f0d1-a41c-4ad8-951b-312faf2cfeb9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:48.992321 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:49.082161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5f0d1-a41c-4ad8-951b-312faf2cfeb9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:49.121122 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf11fdf-515c-44cc-a8bf-0f01681939c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:49.121718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf11fdf-515c-44cc-a8bf-0f01681939c2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:49.121993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf11fdf-515c-44cc-a8bf-0f01681939c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:49.122256 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf11fdf-515c-44cc-a8bf-0f01681939c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:49.122916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf11fdf-515c-44cc-a8bf-0f01681939c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:49.123216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf11fdf-515c-44cc-a8bf-0f01681939c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:49.124592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf11fdf-515c-44cc-a8bf-0f01681939c2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:49.132704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 636/1264] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:26:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:49.139686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53ca06-65d0-4437-aa03-c7a8bf476006 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:49.231640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:49.266350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53ca06-65d0-4437-aa03-c7a8bf476006 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:49.304681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c5e38-fb02-4540-9ab4-0eba371d6b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:49.305310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c5e38-fb02-4540-9ab4-0eba371d6b34 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:49.305632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c5e38-fb02-4540-9ab4-0eba371d6b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:49.305849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c5e38-fb02-4540-9ab4-0eba371d6b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:49.316208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c5e38-fb02-4540-9ab4-0eba371d6b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:49.317668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c5e38-fb02-4540-9ab4-0eba371d6b34 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-d00c5e38-fb02-4540-9ab4-0eba371d6b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:26:49.332813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:49.334289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:26:49.335470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:26:49.335470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:49.335470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:49.335470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 629/1265] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:49.340556 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf11fdf-515c-44cc-a8bf-0f01681939c2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:49.429135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:49.449056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf11fdf-515c-44cc-a8bf-0f01681939c2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:49.496084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5eef7-c1f9-453a-a5ce-6bd6c05619cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:49.496697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5eef7-c1f9-453a-a5ce-6bd6c05619cf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:49.496974 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5eef7-c1f9-453a-a5ce-6bd6c05619cf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:49.497240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5eef7-c1f9-453a-a5ce-6bd6c05619cf None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:49.497881 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5eef7-c1f9-453a-a5ce-6bd6c05619cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:49.498183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5eef7-c1f9-453a-a5ce-6bd6c05619cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:49.499566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5eef7-c1f9-453a-a5ce-6bd6c05619cf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:49.511472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 637/1266] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:26:49.520114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c5e38-fb02-4540-9ab4-0eba371d6b34 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:49.627984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c5e38-fb02-4540-9ab4-0eba371d6b34 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:49.650077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:49.667071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957ad17-a037-42a3-9334-2679ab9fd59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:49.667723 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957ad17-a037-42a3-9334-2679ab9fd59b None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:49.668029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957ad17-a037-42a3-9334-2679ab9fd59b None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:49.668299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957ad17-a037-42a3-9334-2679ab9fd59b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:49.676094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957ad17-a037-42a3-9334-2679ab9fd59b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:49.677589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957ad17-a037-42a3-9334-2679ab9fd59b None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-a957ad17-a037-42a3-9334-2679ab9fd59b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:26:49.689190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:49.690320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:49.691511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:49.691511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:49.691511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:26:49.691511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:26:49.691511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:26:49.691511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:26:49.691511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:26:49.691511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:26:49.691511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:26:49.691511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:49.691511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 630/1267] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:49.697399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5eef7-c1f9-453a-a5ce-6bd6c05619cf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:49.804731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5eef7-c1f9-453a-a5ce-6bd6c05619cf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:49.842620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915dc7a-8955-49f7-bab3-a7f50de17dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:49.843219 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915dc7a-8955-49f7-bab3-a7f50de17dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:49.843475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915dc7a-8955-49f7-bab3-a7f50de17dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:49.843723 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915dc7a-8955-49f7-bab3-a7f50de17dd1 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:49.844508 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915dc7a-8955-49f7-bab3-a7f50de17dd1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:49.844809 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915dc7a-8955-49f7-bab3-a7f50de17dd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:49.846212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915dc7a-8955-49f7-bab3-a7f50de17dd1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:49.855094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:49.868265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 638/1268] 10.222.0.62 () {56 vars in 1143 bytes} [Thu Sep 1 16:26:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:49.874309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957ad17-a037-42a3-9334-2679ab9fd59b None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:49.993284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957ad17-a037-42a3-9334-2679ab9fd59b None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:50.030182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b1c1d-31cc-44e1-9797-55ee483434a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:50.030799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b1c1d-31cc-44e1-9797-55ee483434a0 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:50.031072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b1c1d-31cc-44e1-9797-55ee483434a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:50.031334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b1c1d-31cc-44e1-9797-55ee483434a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:50.042479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b1c1d-31cc-44e1-9797-55ee483434a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:50.043934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b1c1d-31cc-44e1-9797-55ee483434a0 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-600b1c1d-31cc-44e1-9797-55ee483434a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:26:50.058680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:50.059793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:26:50.060889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:26:50.060889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:50.060889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:50.060889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 631/1269] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:50.066074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915dc7a-8955-49f7-bab3-a7f50de17dd1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:50.174914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915dc7a-8955-49f7-bab3-a7f50de17dd1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:50.213631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bf95f-1ceb-4320-93b8-919729ef6243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:50.214208 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bf95f-1ceb-4320-93b8-919729ef6243 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:50.214474 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bf95f-1ceb-4320-93b8-919729ef6243 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:50.214742 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bf95f-1ceb-4320-93b8-919729ef6243 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:50.215364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185bf95f-1ceb-4320-93b8-919729ef6243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:50.215660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185bf95f-1ceb-4320-93b8-919729ef6243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:50.217049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185bf95f-1ceb-4320-93b8-919729ef6243 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:50.228941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 639/1270] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:50.235375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b1c1d-31cc-44e1-9797-55ee483434a0 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:50.270829 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:50.358283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b1c1d-31cc-44e1-9797-55ee483434a0 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:50.394715 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd23549-ca18-4fc0-a753-77b77c96ad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:50.395332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd23549-ca18-4fc0-a753-77b77c96ad51 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:50.395609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd23549-ca18-4fc0-a753-77b77c96ad51 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:50.395874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd23549-ca18-4fc0-a753-77b77c96ad51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:50.403004 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd23549-ca18-4fc0-a753-77b77c96ad51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:50.404479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd23549-ca18-4fc0-a753-77b77c96ad51 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-7fd23549-ca18-4fc0-a753-77b77c96ad51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:26:50.415816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:50.416862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:50.417962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:26:50.417962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:26:50.417962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:26:50.417962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:26:50.417962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:26:50.417962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:26:50.417962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:26:50.417962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:50.417962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 632/1271] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:26:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:26:50.423221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bf95f-1ceb-4320-93b8-919729ef6243 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:50.491087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:50.542932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bf95f-1ceb-4320-93b8-919729ef6243 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:50.581759 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b282db2-5a77-4cc3-9293-25f5a0c81956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:50.582367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b282db2-5a77-4cc3-9293-25f5a0c81956 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:50.582642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b282db2-5a77-4cc3-9293-25f5a0c81956 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:50.582900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b282db2-5a77-4cc3-9293-25f5a0c81956 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:50.583610 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b282db2-5a77-4cc3-9293-25f5a0c81956 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:50.583927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b282db2-5a77-4cc3-9293-25f5a0c81956 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:50.585395 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b282db2-5a77-4cc3-9293-25f5a0c81956 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:50.597786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 640/1272] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:26:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:50.608238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd23549-ca18-4fc0-a753-77b77c96ad51 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:50.743927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:50.758125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd23549-ca18-4fc0-a753-77b77c96ad51 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:50.809303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36146f0a-1ab6-4a7a-a361-781fe79c1178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:50.810274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36146f0a-1ab6-4a7a-a361-781fe79c1178 None admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:50.810727 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36146f0a-1ab6-4a7a-a361-781fe79c1178 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:50.811169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36146f0a-1ab6-4a7a-a361-781fe79c1178 None admin] PATH_INFO: `/v3/projects/425048ffd13448cd8fe815333a9b798c/users/6281dcf2927a4ed2870125df8801be78/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:50.814392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36146f0a-1ab6-4a7a-a361-781fe79c1178 None admin] RBAC: Authorizing `identity:create_grant(project_id=425048ffd13448cd8fe815333a9b798c, user_id=6281dcf2927a4ed2870125df8801be78, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:50.816877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36146f0a-1ab6-4a7a-a361-781fe79c1178 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:50.842685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 633/1273] 10.222.0.62 () {58 vars in 1439 bytes} [Thu Sep 1 16:26:50 2022] PUT /identity/v3/projects/425048ffd13448cd8fe815333a9b798c/users/6281dcf2927a4ed2870125df8801be78/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:26:52.787899 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5abcdb-ad70-43d1-a59f-af1561a2ebc9 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:52.788380 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5abcdb-ad70-43d1-a59f-af1561a2ebc9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:52.788485 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5abcdb-ad70-43d1-a59f-af1561a2ebc9 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:52.789779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 641/1274] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:52.795975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e70647-af79-4878-b074-db7a9a626ad4 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:52.796755 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e70647-af79-4878-b074-db7a9a626ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:52.796909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e70647-af79-4878-b074-db7a9a626ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:52.827150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-17e70647-af79-4878-b074-db7a9a626ad4 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:52.828373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-17e70647-af79-4878-b074-db7a9a626ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:52.903008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 634/1275] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:26:53.182576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5abcdb-ad70-43d1-a59f-af1561a2ebc9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:53.296088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5abcdb-ad70-43d1-a59f-af1561a2ebc9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:53.319575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:53.337045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610953f-6490-4f86-9fee-50d42b2ede9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:53.337678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610953f-6490-4f86-9fee-50d42b2ede9a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:53.337949 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610953f-6490-4f86-9fee-50d42b2ede9a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:53.338212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610953f-6490-4f86-9fee-50d42b2ede9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:53.348909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610953f-6490-4f86-9fee-50d42b2ede9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-1610953f-6490-4f86-9fee-50d42b2ede9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:26:53.353780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:53.354929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:53.355909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:53.355909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:53.355909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:53.355909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:53.355909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:26:53.355909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:26:53.355909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:26:53.355909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:26:53.355909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:26:53.355909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:26:53.355909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:26:53.355909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:53.355909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 642/1276] 10.222.0.62 () {56 vars in 1096 bytes} [Thu Sep 1 16:26:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:53.363622 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e70647-af79-4878-b074-db7a9a626ad4 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:53.454830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:53.484081 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e70647-af79-4878-b074-db7a9a626ad4 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:53.517769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1d282-f223-4f94-b5b1-9d43c2999337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:53.518373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1d282-f223-4f94-b5b1-9d43c2999337 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:53.518644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1d282-f223-4f94-b5b1-9d43c2999337 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:53.518909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1d282-f223-4f94-b5b1-9d43c2999337 None admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:53.519669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1d282-f223-4f94-b5b1-9d43c2999337 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:53.519998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1d282-f223-4f94-b5b1-9d43c2999337 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:53.521417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1d282-f223-4f94-b5b1-9d43c2999337 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:53.529488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 635/1277] 10.222.0.62 () {56 vars in 1099 bytes} [Thu Sep 1 16:26:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:53.539062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610953f-6490-4f86-9fee-50d42b2ede9a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:53.660901 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610953f-6490-4f86-9fee-50d42b2ede9a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:53.698144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3f405-7a27-4469-bf25-651397406ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:53.698748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3f405-7a27-4469-bf25-651397406ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:53.699020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3f405-7a27-4469-bf25-651397406ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:53.699281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3f405-7a27-4469-bf25-651397406ee9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:53.712803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3f405-7a27-4469-bf25-651397406ee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:53.715750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3f405-7a27-4469-bf25-651397406ee9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-30e3f405-7a27-4469-bf25-651397406ee9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:26:53.736327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:53.737509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:26:53.738483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:26:53.738483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:53.738483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:53.738483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 643/1278] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:53.743593 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1d282-f223-4f94-b5b1-9d43c2999337 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:53.850968 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1d282-f223-4f94-b5b1-9d43c2999337 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:53.895200 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e660477-f4f2-4283-b48d-6c7fdef87546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:53.895834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e660477-f4f2-4283-b48d-6c7fdef87546 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:53.896136 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e660477-f4f2-4283-b48d-6c7fdef87546 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:53.896403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e660477-f4f2-4283-b48d-6c7fdef87546 None admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:53.897043 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e660477-f4f2-4283-b48d-6c7fdef87546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:53.897336 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e660477-f4f2-4283-b48d-6c7fdef87546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:53.898713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e660477-f4f2-4283-b48d-6c7fdef87546 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:53.911360 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 636/1279] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:53.917879 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3f405-7a27-4469-bf25-651397406ee9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:53.968735 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:54.046599 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3f405-7a27-4469-bf25-651397406ee9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:54.083740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab88ed6-4892-44fa-944c-8e2bf27420f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:54.084458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab88ed6-4892-44fa-944c-8e2bf27420f3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:54.084731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab88ed6-4892-44fa-944c-8e2bf27420f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:54.084999 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab88ed6-4892-44fa-944c-8e2bf27420f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:54.092665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab88ed6-4892-44fa-944c-8e2bf27420f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:54.094097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab88ed6-4892-44fa-944c-8e2bf27420f3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-5ab88ed6-4892-44fa-944c-8e2bf27420f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:26:54.105349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:54.106323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:54.107392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:54.107392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:54.107392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:26:54.107392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:26:54.107392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:26:54.107392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:26:54.107392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:26:54.107392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:26:54.107392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:26:54.107392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:54.107392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 644/1280] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:26:54.113530 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e660477-f4f2-4283-b48d-6c7fdef87546 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:54.119500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:54.247302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e660477-f4f2-4283-b48d-6c7fdef87546 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:54.286151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95a404-cbbe-47ee-a363-57e0a6a698e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:54.286772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95a404-cbbe-47ee-a363-57e0a6a698e7 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:54.287027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95a404-cbbe-47ee-a363-57e0a6a698e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:54.287307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95a404-cbbe-47ee-a363-57e0a6a698e7 None admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:54.288048 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95a404-cbbe-47ee-a363-57e0a6a698e7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:54.288340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95a404-cbbe-47ee-a363-57e0a6a698e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:54.289782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95a404-cbbe-47ee-a363-57e0a6a698e7 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:54.309696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 637/1281] 10.222.0.62 () {56 vars in 1143 bytes} [Thu Sep 1 16:26:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:54.315634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab88ed6-4892-44fa-944c-8e2bf27420f3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:54.412754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:54.425147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab88ed6-4892-44fa-944c-8e2bf27420f3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:54.465707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958be50-4cec-4d85-b74b-56c610368886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:54.466594 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958be50-4cec-4d85-b74b-56c610368886 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:54.466989 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958be50-4cec-4d85-b74b-56c610368886 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:54.467368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958be50-4cec-4d85-b74b-56c610368886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:54.542288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958be50-4cec-4d85-b74b-56c610368886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:54.543653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958be50-4cec-4d85-b74b-56c610368886 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:54.641162 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:54.657959 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 645/1282] 10.222.0.62 () {58 vars in 1282 bytes} [Thu Sep 1 16:26:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:26:54.664521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95a404-cbbe-47ee-a363-57e0a6a698e7 None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:54.735640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:54.774522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95a404-cbbe-47ee-a363-57e0a6a698e7 None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:54.808260 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac127e44-e3e4-47af-994d-52c0e5f927df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:54.808821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac127e44-e3e4-47af-994d-52c0e5f927df None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:54.809096 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac127e44-e3e4-47af-994d-52c0e5f927df None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:54.809362 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac127e44-e3e4-47af-994d-52c0e5f927df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:54.819225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac127e44-e3e4-47af-994d-52c0e5f927df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:54.820703 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac127e44-e3e4-47af-994d-52c0e5f927df None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-ac127e44-e3e4-47af-994d-52c0e5f927df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:26:54.835359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:54.836462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:26:54.837827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:26:54.837827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:26:54.837827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:54.837827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 638/1283] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:54.842534 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958be50-4cec-4d85-b74b-56c610368886 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:54.948188 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958be50-4cec-4d85-b74b-56c610368886 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:54.987125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b68d00-c6f8-470a-bf44-83c7ae493e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:54.987718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b68d00-c6f8-470a-bf44-83c7ae493e1e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:54.988019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b68d00-c6f8-470a-bf44-83c7ae493e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:54.988283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b68d00-c6f8-470a-bf44-83c7ae493e1e None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:54.988915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b68d00-c6f8-470a-bf44-83c7ae493e1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:54.989208 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b68d00-c6f8-470a-bf44-83c7ae493e1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:54.990562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b68d00-c6f8-470a-bf44-83c7ae493e1e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:55.001688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 646/1284] 10.222.0.62 () {56 vars in 1109 bytes} [Thu Sep 1 16:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:55.010869 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac127e44-e3e4-47af-994d-52c0e5f927df None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:55.133716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac127e44-e3e4-47af-994d-52c0e5f927df None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:55.147542 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:55.178066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5f60b-242d-4098-8d9c-0476ed06333c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:55.178879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5f60b-242d-4098-8d9c-0476ed06333c None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:55.179136 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5f60b-242d-4098-8d9c-0476ed06333c None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:55.179397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5f60b-242d-4098-8d9c-0476ed06333c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:55.187487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5f60b-242d-4098-8d9c-0476ed06333c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:55.188980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5f60b-242d-4098-8d9c-0476ed06333c None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application [None req-9ed5f60b-242d-4098-8d9c-0476ed06333c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:26:55.201582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:55.202614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:55.203639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:26:55.203639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:26:55.203639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:26:55.203639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:26:55.203639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:26:55.203639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:26:55.203639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:26:55.203639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: ERROR keystone.server.flask.application Sep 01 16:26:55.203639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 639/1285] 10.222.0.62 () {56 vars in 1111 bytes} [Thu Sep 1 16:26:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:55.209279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b68d00-c6f8-470a-bf44-83c7ae493e1e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:55.262954 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:55.320895 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b68d00-c6f8-470a-bf44-83c7ae493e1e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:55.356796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de1889-5d08-434f-b681-2813e156b156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:55.357405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de1889-5d08-434f-b681-2813e156b156 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:55.357676 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de1889-5d08-434f-b681-2813e156b156 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:55.357942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de1889-5d08-434f-b681-2813e156b156 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:55.358627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de1889-5d08-434f-b681-2813e156b156 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:55.358923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de1889-5d08-434f-b681-2813e156b156 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:55.360387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de1889-5d08-434f-b681-2813e156b156 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:55.373501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 647/1286] 10.222.0.62 () {56 vars in 1148 bytes} [Thu Sep 1 16:26:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:55.380015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5f60b-242d-4098-8d9c-0476ed06333c None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:55.484626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5f60b-242d-4098-8d9c-0476ed06333c None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:55.523134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf3820-605f-4313-87b9-071818d402a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:55.523717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf3820-605f-4313-87b9-071818d402a8 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:55.524018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf3820-605f-4313-87b9-071818d402a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:55.524253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf3820-605f-4313-87b9-071818d402a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:55.524990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf3820-605f-4313-87b9-071818d402a8 None admin] RBAC: Adding query filter params (role.id=ae7ca58f44a54413bcd33ee40caf4fe1, scope.project.id=425048ffd13448cd8fe815333a9b798c, user.id=6281dcf2927a4ed2870125df8801be78) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:55.525288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf3820-605f-4313-87b9-071818d402a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:55.526828 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf3820-605f-4313-87b9-071818d402a8 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:55.533024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 640/1287] 10.222.0.62 () {56 vars in 1374 bytes} [Thu Sep 1 16:26:55 2022] GET /identity/v3/role_assignments?user.id=6281dcf2927a4ed2870125df8801be78&scope.project.id=425048ffd13448cd8fe815333a9b798c&role.id=ae7ca58f44a54413bcd33ee40caf4fe1 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:57.493293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ee2b2-a896-4551-8487-83307804ff07 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:57.493872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ee2b2-a896-4551-8487-83307804ff07 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:57.493872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ee2b2-a896-4551-8487-83307804ff07 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:57.496011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 648/1288] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:26:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:57.504723 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b72f63-4f83-480a-8b61-b79460d0d6cb None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:57.505466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b72f63-4f83-480a-8b61-b79460d0d6cb None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:57.505643 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b72f63-4f83-480a-8b61-b79460d0d6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:57.528181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-79b72f63-4f83-480a-8b61-b79460d0d6cb None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:57.529397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-79b72f63-4f83-480a-8b61-b79460d0d6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:57.563179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:57.603265 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 641/1289] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:26:57.869821 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ee2b2-a896-4551-8487-83307804ff07 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:57.998211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ee2b2-a896-4551-8487-83307804ff07 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:58.009160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:58.034891 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a52e5-ed37-48ee-aab2-b3010b9d2c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:58.035444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a52e5-ed37-48ee-aab2-b3010b9d2c2a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:58.035721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a52e5-ed37-48ee-aab2-b3010b9d2c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:58.036013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a52e5-ed37-48ee-aab2-b3010b9d2c2a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:58.036588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a52e5-ed37-48ee-aab2-b3010b9d2c2a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:58.037939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a52e5-ed37-48ee-aab2-b3010b9d2c2a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-3a5a52e5-ed37-48ee-aab2-b3010b9d2c2a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:26:58.047273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:26:58.048514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:26:58.049389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 649/1290] 10.222.0.62 () {56 vars in 1117 bytes} [Thu Sep 1 16:26:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:26:58.054671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b72f63-4f83-480a-8b61-b79460d0d6cb None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:58.166486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b72f63-4f83-480a-8b61-b79460d0d6cb None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:58.203490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f587e8-3a67-4a57-ae1c-d088e407827f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:58.204134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f587e8-3a67-4a57-ae1c-d088e407827f None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:58.204416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f587e8-3a67-4a57-ae1c-d088e407827f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:58.204681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f587e8-3a67-4a57-ae1c-d088e407827f None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:58.205312 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f587e8-3a67-4a57-ae1c-d088e407827f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:58.205623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f587e8-3a67-4a57-ae1c-d088e407827f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:58.207345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f587e8-3a67-4a57-ae1c-d088e407827f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:58.212769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 642/1291] 10.222.0.62 () {56 vars in 1116 bytes} [Thu Sep 1 16:26:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:26:58.219749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a52e5-ed37-48ee-aab2-b3010b9d2c2a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:58.342305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a52e5-ed37-48ee-aab2-b3010b9d2c2a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:58.379897 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1bd37-46a5-4d6d-8515-ccf47373ef37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:58.380510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1bd37-46a5-4d6d-8515-ccf47373ef37 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:58.380785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1bd37-46a5-4d6d-8515-ccf47373ef37 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:58.381054 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1bd37-46a5-4d6d-8515-ccf47373ef37 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:58.381680 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe1bd37-46a5-4d6d-8515-ccf47373ef37 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:26:58.381979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe1bd37-46a5-4d6d-8515-ccf47373ef37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:58.383378 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe1bd37-46a5-4d6d-8515-ccf47373ef37 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:58.389477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 650/1292] 10.222.0.62 () {56 vars in 1116 bytes} [Thu Sep 1 16:26:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:27:00.264578 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fab47-72d5-4299-b87d-19efe56d1059 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:00.265163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fab47-72d5-4299-b87d-19efe56d1059 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:00.265163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fab47-72d5-4299-b87d-19efe56d1059 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:00.266542 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 643/1293] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:00.273037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708e638-7471-4676-9ec8-3ef3edc59205 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:00.273791 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708e638-7471-4676-9ec8-3ef3edc59205 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:00.273912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708e638-7471-4676-9ec8-3ef3edc59205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:00.301689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5708e638-7471-4676-9ec8-3ef3edc59205 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:00.302902 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5708e638-7471-4676-9ec8-3ef3edc59205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:00.334017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:00.374715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 651/1294] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:27:00.638356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fab47-72d5-4299-b87d-19efe56d1059 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:00.682994 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:00.761203 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fab47-72d5-4299-b87d-19efe56d1059 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:00.800759 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f17bf-b9d6-4ffe-a157-5131198177bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:00.801378 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f17bf-b9d6-4ffe-a157-5131198177bb None admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:00.801651 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f17bf-b9d6-4ffe-a157-5131198177bb None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:00.801924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f17bf-b9d6-4ffe-a157-5131198177bb None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:00.802598 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f17bf-b9d6-4ffe-a157-5131198177bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:00.804057 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f17bf-b9d6-4ffe-a157-5131198177bb None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:00.820724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 644/1295] 10.222.0.62 () {60 vars in 1141 bytes} [Thu Sep 1 16:27:00 2022] POST /identity/v3/services => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:27:02.732370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f8e8c-962b-4c1a-bf1e-aa8ff175fdf1 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:02.733249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f8e8c-962b-4c1a-bf1e-aa8ff175fdf1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:02.733249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f8e8c-962b-4c1a-bf1e-aa8ff175fdf1 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:02.735374 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 652/1296] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:27:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:02.740836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719061a-db34-4cda-a777-7fff12911790 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:02.741248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719061a-db34-4cda-a777-7fff12911790 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:02.741337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719061a-db34-4cda-a777-7fff12911790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:02.762243 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e719061a-db34-4cda-a777-7fff12911790 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:02.763444 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e719061a-db34-4cda-a777-7fff12911790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:02.840224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 645/1297] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:02 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:27:03.107285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f8e8c-962b-4c1a-bf1e-aa8ff175fdf1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:03.232497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f8e8c-962b-4c1a-bf1e-aa8ff175fdf1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:03.269283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac9612-a63e-4846-9d8d-bb41332d601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:03.269893 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac9612-a63e-4846-9d8d-bb41332d601f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:03.270166 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac9612-a63e-4846-9d8d-bb41332d601f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:03.270419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac9612-a63e-4846-9d8d-bb41332d601f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:03.270993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac9612-a63e-4846-9d8d-bb41332d601f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:03.272376 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac9612-a63e-4846-9d8d-bb41332d601f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c1ac9612-a63e-4846-9d8d-bb41332d601f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:27:03.281982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:27:03.283149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:27:03.284889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 653/1298] 10.222.0.62 () {56 vars in 1117 bytes} [Thu Sep 1 16:27:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:27:03.292867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719061a-db34-4cda-a777-7fff12911790 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:03.322478 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:03.409743 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719061a-db34-4cda-a777-7fff12911790 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:03.449902 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c573576-1387-4c06-8d2e-850717c76b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:03.450499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c573576-1387-4c06-8d2e-850717c76b7c None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:03.450777 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c573576-1387-4c06-8d2e-850717c76b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:03.451041 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c573576-1387-4c06-8d2e-850717c76b7c None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:03.451670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c573576-1387-4c06-8d2e-850717c76b7c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:27:03.452000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c573576-1387-4c06-8d2e-850717c76b7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:03.453414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c573576-1387-4c06-8d2e-850717c76b7c None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:03.459075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 646/1299] 10.222.0.62 () {56 vars in 1116 bytes} [Thu Sep 1 16:27:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:27:03.465991 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac9612-a63e-4846-9d8d-bb41332d601f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:03.501527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:03.573454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac9612-a63e-4846-9d8d-bb41332d601f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:03.610556 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559d758-afdb-4e39-a943-d54019706b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:03.611146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559d758-afdb-4e39-a943-d54019706b6a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:03.611430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559d758-afdb-4e39-a943-d54019706b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:03.611694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559d758-afdb-4e39-a943-d54019706b6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:03.612423 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559d758-afdb-4e39-a943-d54019706b6a None admin] RBAC: Adding query filter params (interface=public, service_id=965b88286b094a26b74cd93a4fa4e754, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:27:03.612736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559d758-afdb-4e39-a943-d54019706b6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:03.614088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559d758-afdb-4e39-a943-d54019706b6a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:03.619781 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 654/1300] 10.222.0.62 () {56 vars in 1251 bytes} [Thu Sep 1 16:27:03 2022] GET /identity/v3/endpoints?service_id=965b88286b094a26b74cd93a4fa4e754&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:27:03.625785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c573576-1387-4c06-8d2e-850717c76b7c None admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:03.710680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:03.732157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c573576-1387-4c06-8d2e-850717c76b7c None admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:03.769807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76adf2c-f82f-43cb-a7aa-a30556e6e186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:03.770402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76adf2c-f82f-43cb-a7aa-a30556e6e186 None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:03.770656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76adf2c-f82f-43cb-a7aa-a30556e6e186 None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:03.770920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76adf2c-f82f-43cb-a7aa-a30556e6e186 None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:03.771511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76adf2c-f82f-43cb-a7aa-a30556e6e186 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:03.772923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76adf2c-f82f-43cb-a7aa-a30556e6e186 None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:03.780159 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 647/1301] 10.222.0.62 () {56 vars in 1087 bytes} [Thu Sep 1 16:27:03 2022] GET /identity/v3/services => generated 2191 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:27:05.686640 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df9d07-2489-4eda-97a2-f3bf029f95d7 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:05.687550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df9d07-2489-4eda-97a2-f3bf029f95d7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:05.687550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df9d07-2489-4eda-97a2-f3bf029f95d7 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:05.689769 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 655/1302] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:27:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:05.696456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c223ae8-ffa6-49f7-a688-844ef4f46ef1 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:05.696996 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c223ae8-ffa6-49f7-a688-844ef4f46ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:05.696996 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c223ae8-ffa6-49f7-a688-844ef4f46ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:05.717874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-2c223ae8-ffa6-49f7-a688-844ef4f46ef1 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:05.719154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2c223ae8-ffa6-49f7-a688-844ef4f46ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:05.797637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 648/1303] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:27:06.057087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df9d07-2489-4eda-97a2-f3bf029f95d7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:06.168473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:06.181515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df9d07-2489-4eda-97a2-f3bf029f95d7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:06.216779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101d52c-8b08-45fb-84fe-94c0dbf76241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:06.217402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101d52c-8b08-45fb-84fe-94c0dbf76241 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:06.217674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101d52c-8b08-45fb-84fe-94c0dbf76241 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:06.217935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101d52c-8b08-45fb-84fe-94c0dbf76241 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:06.218527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101d52c-8b08-45fb-84fe-94c0dbf76241 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:06.220010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101d52c-8b08-45fb-84fe-94c0dbf76241 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b101d52c-8b08-45fb-84fe-94c0dbf76241 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:27:06.228987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:27:06.230135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 01 16:27:06.231479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 656/1304] 10.222.0.62 () {56 vars in 1117 bytes} [Thu Sep 1 16:27:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:27:06.239269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c223ae8-ffa6-49f7-a688-844ef4f46ef1 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:06.341982 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:06.376180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c223ae8-ffa6-49f7-a688-844ef4f46ef1 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:06.416780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ba4d4-c9ef-464c-8b7b-22b8ead7fd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:06.417385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ba4d4-c9ef-464c-8b7b-22b8ead7fd2f None admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:06.417656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ba4d4-c9ef-464c-8b7b-22b8ead7fd2f None admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:06.417920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ba4d4-c9ef-464c-8b7b-22b8ead7fd2f None admin] PATH_INFO: `/v3/services` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:06.418538 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ba4d4-c9ef-464c-8b7b-22b8ead7fd2f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:27:06.418836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ba4d4-c9ef-464c-8b7b-22b8ead7fd2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:06.420272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ba4d4-c9ef-464c-8b7b-22b8ead7fd2f None admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:06.425429 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 649/1305] 10.222.0.62 () {56 vars in 1116 bytes} [Thu Sep 1 16:27:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:27:06.432273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101d52c-8b08-45fb-84fe-94c0dbf76241 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:06.544851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101d52c-8b08-45fb-84fe-94c0dbf76241 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:06.580464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d2af8-4d95-4b17-94cd-2e12ee10f2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:06.581362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d2af8-4d95-4b17-94cd-2e12ee10f2fc None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:06.581638 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d2af8-4d95-4b17-94cd-2e12ee10f2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:06.581903 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d2af8-4d95-4b17-94cd-2e12ee10f2fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:06.582552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d2af8-4d95-4b17-94cd-2e12ee10f2fc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:06.583995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d2af8-4d95-4b17-94cd-2e12ee10f2fc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:06.606389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 657/1306] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:27:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:27:16.422297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=30 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:16.601823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:11.711738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d819c-2a18-4b7b-b3dc-53f09278f322 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:11.712182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d819c-2a18-4b7b-b3dc-53f09278f322 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:11.712301 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d819c-2a18-4b7b-b3dc-53f09278f322 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:11.714983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 650/1307] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:28:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:11.720702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5c57d-1957-4c63-9361-c707231d6d93 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:11.721203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5c57d-1957-4c63-9361-c707231d6d93 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:11.721203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5c57d-1957-4c63-9361-c707231d6d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:11.900223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9dd5c57d-1957-4c63-9361-c707231d6d93 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:11.903068 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9dd5c57d-1957-4c63-9361-c707231d6d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:12.053006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 658/1308] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:12.351238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a9c74-389f-45a2-9a58-6eb29ab1652f None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:12.351880 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a9c74-389f-45a2-9a58-6eb29ab1652f None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:12.352386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a9c74-389f-45a2-9a58-6eb29ab1652f None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:12.354089 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 651/1309] 10.222.0.62 () {54 vars in 965 bytes} [Thu Sep 1 16:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:12.360902 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d9938-a425-42af-90ef-9e9c9036eba2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:12.361221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d9938-a425-42af-90ef-9e9c9036eba2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:12.361479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d9938-a425-42af-90ef-9e9c9036eba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:12.448703 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-338d9938-a425-42af-90ef-9e9c9036eba2 None None] MFA Rules not processed for user `0718a95189434a5ea901c40bc868bea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:12.449935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-338d9938-a425-42af-90ef-9e9c9036eba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:12.473217 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:12.543388 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 659/1310] 10.222.0.62 () {58 vars in 1062 bytes} [Thu Sep 1 16:28:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:28:12.546193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a9c74-389f-45a2-9a58-6eb29ab1652f None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:12.548152 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-638a9c74-389f-45a2-9a58-6eb29ab1652f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:12.669836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a9c74-389f-45a2-9a58-6eb29ab1652f None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:12.753947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4a7ea-47dd-492a-a285-49c1e6ca0aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:12.754585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4a7ea-47dd-492a-a285-49c1e6ca0aef service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:12.754757 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4a7ea-47dd-492a-a285-49c1e6ca0aef service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:12.754999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4a7ea-47dd-492a-a285-49c1e6ca0aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:12.758427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0ef4a7ea-47dd-492a-a285-49c1e6ca0aef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:12.819912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:12.881028 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4a7ea-47dd-492a-a285-49c1e6ca0aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:12.882833 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4a7ea-47dd-492a-a285-49c1e6ca0aef service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:12.906903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 652/1311] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:15.220296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55727b-242d-4157-877e-4714efa2ac3c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:15.220890 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55727b-242d-4157-877e-4714efa2ac3c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:15.220890 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55727b-242d-4157-877e-4714efa2ac3c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:15.222185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 660/1312] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:15.228083 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115fea8-6647-4e03-a055-5580aa33452b None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:15.228640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115fea8-6647-4e03-a055-5580aa33452b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:15.228640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115fea8-6647-4e03-a055-5580aa33452b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:15.309696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-7115fea8-6647-4e03-a055-5580aa33452b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:15.310929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7115fea8-6647-4e03-a055-5580aa33452b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:15.410904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 653/1313] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:15.698191 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ad40b-393b-4af9-9c7f-b132c516fe28 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:15.698606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ad40b-393b-4af9-9c7f-b132c516fe28 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:15.698883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ad40b-393b-4af9-9c7f-b132c516fe28 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:15.700349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 661/1314] 10.222.0.62 () {54 vars in 965 bytes} [Thu Sep 1 16:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:15.706923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e031c-6820-4888-ab4b-7f7127f9f0df None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:15.707209 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e031c-6820-4888-ab4b-7f7127f9f0df None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:15.707478 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e031c-6820-4888-ab4b-7f7127f9f0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:15.719825 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:15.791335 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-f75e031c-6820-4888-ab4b-7f7127f9f0df None None] MFA Rules not processed for user `0718a95189434a5ea901c40bc868bea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:15.792587 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f75e031c-6820-4888-ab4b-7f7127f9f0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:15.871128 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 654/1315] 10.222.0.62 () {58 vars in 1062 bytes} [Thu Sep 1 16:28:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:15.880140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ad40b-393b-4af9-9c7f-b132c516fe28 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:15.881750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6c2ad40b-393b-4af9-9c7f-b132c516fe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:15.997616 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ad40b-393b-4af9-9c7f-b132c516fe28 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:16.069897 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:16.072761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084861a6-c6ed-44fe-93d4-ef6d73ee4e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:16.073582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084861a6-c6ed-44fe-93d4-ef6d73ee4e4f service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:16.074042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084861a6-c6ed-44fe-93d4-ef6d73ee4e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:16.074511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084861a6-c6ed-44fe-93d4-ef6d73ee4e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:16.076158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-084861a6-c6ed-44fe-93d4-ef6d73ee4e4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:16.196137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084861a6-c6ed-44fe-93d4-ef6d73ee4e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:16.197589 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084861a6-c6ed-44fe-93d4-ef6d73ee4e4f service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:16.216801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 662/1316] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:18.643583 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7391d-2ad0-4e57-a56c-4d24e1f4bf3f None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:18.644596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7391d-2ad0-4e57-a56c-4d24e1f4bf3f None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:18.644596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7391d-2ad0-4e57-a56c-4d24e1f4bf3f None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:18.646613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 655/1317] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:28:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:18.655346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb677327-e237-4150-b249-b0b915250456 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:18.655771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb677327-e237-4150-b249-b0b915250456 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:18.655880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb677327-e237-4150-b249-b0b915250456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:18.734066 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-eb677327-e237-4150-b249-b0b915250456 None None] MFA Rules not processed for user `ab56ac1625ce49dfba98e1bb018da72d`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:18.735282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eb677327-e237-4150-b249-b0b915250456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:18.801692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:18.850134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 663/1318] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:19.141350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7391d-2ad0-4e57-a56c-4d24e1f4bf3f None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:19.142955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b4a7391d-2ad0-4e57-a56c-4d24e1f4bf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:19.256407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7391d-2ad0-4e57-a56c-4d24e1f4bf3f None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:19.259515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:19.338497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521fc90-d4f0-4fc7-8943-8a0e2c2f9a61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:19.339118 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521fc90-d4f0-4fc7-8943-8a0e2c2f9a61 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:19.339392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521fc90-d4f0-4fc7-8943-8a0e2c2f9a61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:19.339655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521fc90-d4f0-4fc7-8943-8a0e2c2f9a61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:19.341288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3521fc90-d4f0-4fc7-8943-8a0e2c2f9a61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:19.455673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521fc90-d4f0-4fc7-8943-8a0e2c2f9a61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:19.457065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521fc90-d4f0-4fc7-8943-8a0e2c2f9a61 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:19.475433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 656/1319] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:19 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:21.977194 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253081ff-0221-490c-9a69-ce899ca6d88d None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:21.977815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253081ff-0221-490c-9a69-ce899ca6d88d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:21.977815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253081ff-0221-490c-9a69-ce899ca6d88d None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:21.979071 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 664/1320] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:21.985494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01d397-0f3c-403a-988f-0e669ead9128 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:21.985960 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01d397-0f3c-403a-988f-0e669ead9128 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:21.986063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01d397-0f3c-403a-988f-0e669ead9128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:22.038010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:22.065973 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-ab01d397-0f3c-403a-988f-0e669ead9128 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:22.067193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ab01d397-0f3c-403a-988f-0e669ead9128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:22.168753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 657/1321] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:22.464843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253081ff-0221-490c-9a69-ce899ca6d88d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:22.467026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-253081ff-0221-490c-9a69-ce899ca6d88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:22.586223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253081ff-0221-490c-9a69-ce899ca6d88d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:22.660776 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665fcf4-10ae-489e-8215-9b16458f88f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:22.661366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665fcf4-10ae-489e-8215-9b16458f88f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:22.661671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665fcf4-10ae-489e-8215-9b16458f88f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:22.661943 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665fcf4-10ae-489e-8215-9b16458f88f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:22.663599 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b665fcf4-10ae-489e-8215-9b16458f88f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:22.671406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:22.781843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665fcf4-10ae-489e-8215-9b16458f88f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:22.783307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665fcf4-10ae-489e-8215-9b16458f88f7 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:22.802673 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 665/1322] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:25.384532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d407dd-7e14-4860-8c09-56731ae1129b None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:25.385082 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d407dd-7e14-4860-8c09-56731ae1129b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:25.385082 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d407dd-7e14-4860-8c09-56731ae1129b None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:25.386558 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 658/1323] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:25.391837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173b6e4-0633-4b78-8882-337ece257bb6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:25.392311 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173b6e4-0633-4b78-8882-337ece257bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:25.392524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173b6e4-0633-4b78-8882-337ece257bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:25.461694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d173b6e4-0633-4b78-8882-337ece257bb6 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:25.462909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d173b6e4-0633-4b78-8882-337ece257bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:25.522693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:25.563731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 666/1324] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:25.854225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d407dd-7e14-4860-8c09-56731ae1129b None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:25.855871 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-55d407dd-7e14-4860-8c09-56731ae1129b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:25.968864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d407dd-7e14-4860-8c09-56731ae1129b None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:25.981738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:26.046397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea9e1e-548d-4a7f-b095-a3ff82d88e1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:26.046900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea9e1e-548d-4a7f-b095-a3ff82d88e1b service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:26.047182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea9e1e-548d-4a7f-b095-a3ff82d88e1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:26.047447 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea9e1e-548d-4a7f-b095-a3ff82d88e1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:26.049089 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e1ea9e1e-548d-4a7f-b095-a3ff82d88e1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:26.163936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea9e1e-548d-4a7f-b095-a3ff82d88e1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:26.165400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea9e1e-548d-4a7f-b095-a3ff82d88e1b service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:26.184522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 659/1325] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:28.179242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd886de-5324-4cc3-b734-70fa72ff76e0 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:28.179784 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd886de-5324-4cc3-b734-70fa72ff76e0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:28.180089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd886de-5324-4cc3-b734-70fa72ff76e0 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:28.181547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 667/1326] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:28.186947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33deea32-4189-418b-a5fb-16ddb477f85d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:28.187398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33deea32-4189-418b-a5fb-16ddb477f85d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:28.187615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33deea32-4189-418b-a5fb-16ddb477f85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:28.240143 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:28.259754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-33deea32-4189-418b-a5fb-16ddb477f85d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:28.261024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-33deea32-4189-418b-a5fb-16ddb477f85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:28.360221 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 660/1327] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:28.638565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd886de-5324-4cc3-b734-70fa72ff76e0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:28.640201 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8cd886de-5324-4cc3-b734-70fa72ff76e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:28.754931 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd886de-5324-4cc3-b734-70fa72ff76e0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:28.831394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61afda-8629-470c-8e28-1831c15c7b75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:28.832040 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61afda-8629-470c-8e28-1831c15c7b75 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:28.832288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61afda-8629-470c-8e28-1831c15c7b75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:28.832576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61afda-8629-470c-8e28-1831c15c7b75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:28.834171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-de61afda-8629-470c-8e28-1831c15c7b75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:28.852368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:28.952657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61afda-8629-470c-8e28-1831c15c7b75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:28.954477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61afda-8629-470c-8e28-1831c15c7b75 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:28.974103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 668/1328] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:28:30.953892 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a2813-fdd4-49bf-a1b0-a4004e3ba78f None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:30.954860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a2813-fdd4-49bf-a1b0-a4004e3ba78f None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:30.954860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a2813-fdd4-49bf-a1b0-a4004e3ba78f None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:30.957183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 661/1329] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:28:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:30.965870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eef4e8-4e76-4f3d-bb66-c5eb49c098be None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:30.966317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eef4e8-4e76-4f3d-bb66-c5eb49c098be None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:30.966738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eef4e8-4e76-4f3d-bb66-c5eb49c098be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:31.040043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-33eef4e8-4e76-4f3d-bb66-c5eb49c098be None None] MFA Rules not processed for user `ab56ac1625ce49dfba98e1bb018da72d`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:31.041534 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-33eef4e8-4e76-4f3d-bb66-c5eb49c098be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:31.113161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:31.145017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 669/1330] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:31.427758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a2813-fdd4-49bf-a1b0-a4004e3ba78f None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:31.429454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1e9a2813-fdd4-49bf-a1b0-a4004e3ba78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:31.543475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a2813-fdd4-49bf-a1b0-a4004e3ba78f None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:31.564034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:31.621690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806219ec-8c32-45aa-9cee-3fdb76f4ff91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:31.622312 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806219ec-8c32-45aa-9cee-3fdb76f4ff91 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:31.622582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806219ec-8c32-45aa-9cee-3fdb76f4ff91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:31.622848 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806219ec-8c32-45aa-9cee-3fdb76f4ff91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:31.624480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-806219ec-8c32-45aa-9cee-3fdb76f4ff91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:31.741989 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806219ec-8c32-45aa-9cee-3fdb76f4ff91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:31.743443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806219ec-8c32-45aa-9cee-3fdb76f4ff91 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:31.762679 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 662/1331] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:34.065415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e6288-271d-4088-8569-5d970e0c8b9d None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:34.065963 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e6288-271d-4088-8569-5d970e0c8b9d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:34.065963 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e6288-271d-4088-8569-5d970e0c8b9d None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:34.067310 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 670/1332] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:34.073795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03667b-0370-4ff2-b337-792b55225473 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:34.074482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03667b-0370-4ff2-b337-792b55225473 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:34.074675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03667b-0370-4ff2-b337-792b55225473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:34.151040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:34.154220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6a03667b-0370-4ff2-b337-792b55225473 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:34.155984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6a03667b-0370-4ff2-b337-792b55225473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:34.261156 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 663/1333] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:34.549299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e6288-271d-4088-8569-5d970e0c8b9d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:34.551729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ee1e6288-271d-4088-8569-5d970e0c8b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:34.667474 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e6288-271d-4088-8569-5d970e0c8b9d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:34.746769 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39534e5-91cc-47d8-9c2e-d2b3c14f8f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:34.747682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39534e5-91cc-47d8-9c2e-d2b3c14f8f65 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:34.748043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39534e5-91cc-47d8-9c2e-d2b3c14f8f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:34.748334 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39534e5-91cc-47d8-9c2e-d2b3c14f8f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:34.749979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c39534e5-91cc-47d8-9c2e-d2b3c14f8f65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:34.780138 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:34.873465 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39534e5-91cc-47d8-9c2e-d2b3c14f8f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:34.874892 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39534e5-91cc-47d8-9c2e-d2b3c14f8f65 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:34.894304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 671/1334] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:36.869892 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627812f-4bad-409a-8ed1-a72d01542200 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:36.870776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627812f-4bad-409a-8ed1-a72d01542200 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:36.870776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627812f-4bad-409a-8ed1-a72d01542200 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:36.873019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 664/1335] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:28:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:36.881827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a9d93-c22e-4aab-bc35-e815962c9058 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:36.882590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a9d93-c22e-4aab-bc35-e815962c9058 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:36.882590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a9d93-c22e-4aab-bc35-e815962c9058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:36.955729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-727a9d93-c22e-4aab-bc35-e815962c9058 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:36.956958 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-727a9d93-c22e-4aab-bc35-e815962c9058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:37.046201 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:37.072046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 672/1336] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:37.354858 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627812f-4bad-409a-8ed1-a72d01542200 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:37.356426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3627812f-4bad-409a-8ed1-a72d01542200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:37.464565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627812f-4bad-409a-8ed1-a72d01542200 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:37.495406 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:37.544058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0d199-52f8-4488-96ea-b63e2e0d153e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:37.544650 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0d199-52f8-4488-96ea-b63e2e0d153e service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:37.544934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0d199-52f8-4488-96ea-b63e2e0d153e service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:37.545196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0d199-52f8-4488-96ea-b63e2e0d153e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:37.546771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-eee0d199-52f8-4488-96ea-b63e2e0d153e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:37.677284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0d199-52f8-4488-96ea-b63e2e0d153e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:37.678636 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0d199-52f8-4488-96ea-b63e2e0d153e service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:37.697608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 665/1337] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:39.954557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7a00f-91d6-4ebd-aacb-f52f6f4d4d9e None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:39.955467 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7a00f-91d6-4ebd-aacb-f52f6f4d4d9e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:39.955467 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7a00f-91d6-4ebd-aacb-f52f6f4d4d9e None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:39.957598 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 673/1338] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:28:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:39.966145 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2369cd-b6c1-4ea4-b0f5-f397d06cf535 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:39.966595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2369cd-b6c1-4ea4-b0f5-f397d06cf535 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:39.966714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2369cd-b6c1-4ea4-b0f5-f397d06cf535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:40.042425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-dd2369cd-b6c1-4ea4-b0f5-f397d06cf535 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:40.043560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-dd2369cd-b6c1-4ea4-b0f5-f397d06cf535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:40.052090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:40.145966 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 666/1339] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:40.426560 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7a00f-91d6-4ebd-aacb-f52f6f4d4d9e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:40.429139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8bf7a00f-91d6-4ebd-aacb-f52f6f4d4d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:40.539484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7a00f-91d6-4ebd-aacb-f52f6f4d4d9e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:40.623066 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b9c57-2234-4901-92c3-a9a53aa0c247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:40.623669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b9c57-2234-4901-92c3-a9a53aa0c247 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:40.623947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b9c57-2234-4901-92c3-a9a53aa0c247 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:40.624237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b9c57-2234-4901-92c3-a9a53aa0c247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:40.625827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5a2b9c57-2234-4901-92c3-a9a53aa0c247 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:40.661230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:40.756723 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b9c57-2234-4901-92c3-a9a53aa0c247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:40.758218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b9c57-2234-4901-92c3-a9a53aa0c247 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:40.777845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 674/1340] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:44.705763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2579-8f4a-47f4-9a50-7f6d149aed97 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:44.706346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2579-8f4a-47f4-9a50-7f6d149aed97 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:44.706346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2579-8f4a-47f4-9a50-7f6d149aed97 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:44.707645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 667/1341] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:28:44.713503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a8ed4-0068-47fc-b173-fad0f62b13f4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:44.713939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a8ed4-0068-47fc-b173-fad0f62b13f4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:44.714034 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a8ed4-0068-47fc-b173-fad0f62b13f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:44.784805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dc8a8ed4-0068-47fc-b173-fad0f62b13f4 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:44.786013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dc8a8ed4-0068-47fc-b173-fad0f62b13f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:44.875495 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:44.887740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 675/1342] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:45.166322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d2579-8f4a-47f4-9a50-7f6d149aed97 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:45.167853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a64d2579-8f4a-47f4-9a50-7f6d149aed97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:45.278068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d2579-8f4a-47f4-9a50-7f6d149aed97 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:45.320204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:45.358715 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77744057-79d3-44e4-9a69-3a58f1ce4d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:45.359333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77744057-79d3-44e4-9a69-3a58f1ce4d61 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:45.359633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77744057-79d3-44e4-9a69-3a58f1ce4d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:45.359915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77744057-79d3-44e4-9a69-3a58f1ce4d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:45.361553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-77744057-79d3-44e4-9a69-3a58f1ce4d61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:45.474133 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77744057-79d3-44e4-9a69-3a58f1ce4d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:45.475502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77744057-79d3-44e4-9a69-3a58f1ce4d61 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:45.494141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 668/1343] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:48.280906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688b70c-b610-4746-8970-fbb6f70e2128 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:48.281476 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688b70c-b610-4746-8970-fbb6f70e2128 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:48.281476 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688b70c-b610-4746-8970-fbb6f70e2128 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:48.283007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 676/1344] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:48.288455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf07456-aefc-43f6-9784-e53e78288109 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:48.288939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf07456-aefc-43f6-9784-e53e78288109 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:48.289207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf07456-aefc-43f6-9784-e53e78288109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:48.360985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6cf07456-aefc-43f6-9784-e53e78288109 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:48.362190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6cf07456-aefc-43f6-9784-e53e78288109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:48.381478 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:48.467263 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 669/1345] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:48.747320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688b70c-b610-4746-8970-fbb6f70e2128 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:48.748977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c688b70c-b610-4746-8970-fbb6f70e2128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:48.865145 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688b70c-b610-4746-8970-fbb6f70e2128 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:48.943621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08814a42-bfbd-4436-8bde-1e272788637e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:48.944270 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08814a42-bfbd-4436-8bde-1e272788637e service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:48.944704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08814a42-bfbd-4436-8bde-1e272788637e service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:48.944980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08814a42-bfbd-4436-8bde-1e272788637e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:48.946596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-08814a42-bfbd-4436-8bde-1e272788637e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:48.993341 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:49.069939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08814a42-bfbd-4436-8bde-1e272788637e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:49.071534 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08814a42-bfbd-4436-8bde-1e272788637e service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:49.091823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 677/1346] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:53.290832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af127f51-5cb8-4ce2-8277-74798eb75f5f None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:53.291508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af127f51-5cb8-4ce2-8277-74798eb75f5f None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:53.291508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af127f51-5cb8-4ce2-8277-74798eb75f5f None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:53.293050 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 670/1347] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:53.298447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcd9fa-c74b-4069-94fa-cc0ff05c3401 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:53.298929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcd9fa-c74b-4069-94fa-cc0ff05c3401 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:53.299045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcd9fa-c74b-4069-94fa-cc0ff05c3401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:53.373343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-96bcd9fa-c74b-4069-94fa-cc0ff05c3401 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:53.374720 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-96bcd9fa-c74b-4069-94fa-cc0ff05c3401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:53.481546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:53.488044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 678/1348] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:28:53.764341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af127f51-5cb8-4ce2-8277-74798eb75f5f None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:28:53.765889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-af127f51-5cb8-4ce2-8277-74798eb75f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:53.879112 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af127f51-5cb8-4ce2-8277-74798eb75f5f None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:28:53.928061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:53.956514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a2ebe-01e6-4e62-8ac9-8fc7be3ed11b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:28:53.957126 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a2ebe-01e6-4e62-8ac9-8fc7be3ed11b service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:53.957402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a2ebe-01e6-4e62-8ac9-8fc7be3ed11b service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:53.957666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a2ebe-01e6-4e62-8ac9-8fc7be3ed11b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:53.959318 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-044a2ebe-01e6-4e62-8ac9-8fc7be3ed11b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:54.070497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a2ebe-01e6-4e62-8ac9-8fc7be3ed11b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:28:54.071991 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a2ebe-01e6-4e62-8ac9-8fc7be3ed11b service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:28:54.090752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 671/1349] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:28:59.069362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505fa3f-6302-4948-ae3a-ee3407c82d43 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:59.070514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505fa3f-6302-4948-ae3a-ee3407c82d43 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:59.070944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505fa3f-6302-4948-ae3a-ee3407c82d43 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:59.072784 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 679/1350] 10.222.0.62 () {54 vars in 921 bytes} [Thu Sep 1 16:28:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:28:59.079175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a7677-9d15-4987-86e3-041afb5b0afa None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:28:59.079649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a7677-9d15-4987-86e3-041afb5b0afa None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:28:59.082745 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a7677-9d15-4987-86e3-041afb5b0afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:28:59.200246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-d50a7677-9d15-4987-86e3-041afb5b0afa None None] MFA Rules not processed for user `6281dcf2927a4ed2870125df8801be78`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:28:59.207425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d50a7677-9d15-4987-86e3-041afb5b0afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:28:59.236363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:28:59.346544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 672/1351] 10.222.0.62 () {58 vars in 1018 bytes} [Thu Sep 1 16:28:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:03.535182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d5654-7ffd-4a70-b12d-b358dfde536e None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:03.535703 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d5654-7ffd-4a70-b12d-b358dfde536e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:03.535703 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d5654-7ffd-4a70-b12d-b358dfde536e None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:03.536987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 680/1352] 10.222.0.62 () {54 vars in 921 bytes} [Thu Sep 1 16:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:03.542472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54470346-1be1-469d-a944-20dab112a1eb None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:03.542926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54470346-1be1-469d-a944-20dab112a1eb None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:03.543021 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54470346-1be1-469d-a944-20dab112a1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:03.623004 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-54470346-1be1-469d-a944-20dab112a1eb None None] MFA Rules not processed for user `6281dcf2927a4ed2870125df8801be78`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:03.624235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-54470346-1be1-469d-a944-20dab112a1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:03.700230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:03.742255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 673/1353] 10.222.0.62 () {58 vars in 1018 bytes} [Thu Sep 1 16:29:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:05.014774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fffe3-5d45-4075-bcc9-679ab9b548ce None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:05.015313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fffe3-5d45-4075-bcc9-679ab9b548ce None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:05.015313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fffe3-5d45-4075-bcc9-679ab9b548ce None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:05.016579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 681/1354] 10.222.0.62 () {54 vars in 921 bytes} [Thu Sep 1 16:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:05.022009 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f1ffd-9685-4447-9ea4-312186430621 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:05.022420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f1ffd-9685-4447-9ea4-312186430621 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:05.022625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f1ffd-9685-4447-9ea4-312186430621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:05.104746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-330f1ffd-9685-4447-9ea4-312186430621 None None] MFA Rules not processed for user `6281dcf2927a4ed2870125df8801be78`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:05.105957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-330f1ffd-9685-4447-9ea4-312186430621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:05.218280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 674/1355] 10.222.0.62 () {58 vars in 1018 bytes} [Thu Sep 1 16:29:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:08.631536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6be523-85ff-4ee0-ab17-da29f2da1d3c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:08.632268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6be523-85ff-4ee0-ab17-da29f2da1d3c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:08.632268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6be523-85ff-4ee0-ab17-da29f2da1d3c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:08.633468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 682/1356] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:08.638705 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe81dc-b43d-409c-a397-02848e0597a1 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:08.639150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe81dc-b43d-409c-a397-02848e0597a1 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:08.639300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe81dc-b43d-409c-a397-02848e0597a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:08.681939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:08.719911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-64fe81dc-b43d-409c-a397-02848e0597a1 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:08.721139 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-64fe81dc-b43d-409c-a397-02848e0597a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:08.858306 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 675/1357] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:09.157921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec460f-680b-4f9e-a073-0639bb6d3ea9 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:09.158437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec460f-680b-4f9e-a073-0639bb6d3ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:09.158927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec460f-680b-4f9e-a073-0639bb6d3ea9 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:09.160905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 683/1358] 10.222.0.62 () {54 vars in 962 bytes} [Thu Sep 1 16:29:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:09.170106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d7ab8-d57d-40c0-a9dd-4b2ed02c9dfd None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:09.170437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d7ab8-d57d-40c0-a9dd-4b2ed02c9dfd None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:09.170704 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d7ab8-d57d-40c0-a9dd-4b2ed02c9dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:09.250614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-c30d7ab8-d57d-40c0-a9dd-4b2ed02c9dfd None None] MFA Rules not processed for user `2912c02ff6db4cbc97c80cd923238136`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:09.251825 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c30d7ab8-d57d-40c0-a9dd-4b2ed02c9dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:09.305564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:09.402424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 676/1359] 10.222.0.62 () {58 vars in 1059 bytes} [Thu Sep 1 16:29:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:09.410738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec460f-680b-4f9e-a073-0639bb6d3ea9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:09.413282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-73ec460f-680b-4f9e-a073-0639bb6d3ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:09.566271 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec460f-680b-4f9e-a073-0639bb6d3ea9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:09.644343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:09.687978 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de9545-58ff-438b-998f-896e78cfb5ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:09.688497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de9545-58ff-438b-998f-896e78cfb5ff service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:09.688775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de9545-58ff-438b-998f-896e78cfb5ff service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:09.689046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de9545-58ff-438b-998f-896e78cfb5ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:09.690657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-37de9545-58ff-438b-998f-896e78cfb5ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:09.798303 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de9545-58ff-438b-998f-896e78cfb5ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:09.800315 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de9545-58ff-438b-998f-896e78cfb5ff service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:09.819640 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 684/1360] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:11.939487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eecae-76e5-4ce1-a9d4-029f00a31efe None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:11.940052 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eecae-76e5-4ce1-a9d4-029f00a31efe None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:11.940052 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eecae-76e5-4ce1-a9d4-029f00a31efe None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:11.941512 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 677/1361] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:11.946754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303123b-bd25-4fe5-8593-47c1816626ee None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:11.947199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303123b-bd25-4fe5-8593-47c1816626ee None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:11.947291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303123b-bd25-4fe5-8593-47c1816626ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:12.022020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9303123b-bd25-4fe5-8593-47c1816626ee None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:12.023402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9303123b-bd25-4fe5-8593-47c1816626ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:12.065272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:12.129389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 685/1362] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:12.412878 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642eecae-76e5-4ce1-a9d4-029f00a31efe None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:12.414405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-642eecae-76e5-4ce1-a9d4-029f00a31efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:12.602325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642eecae-76e5-4ce1-a9d4-029f00a31efe None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:12.612176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:12.722782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b5ef1-c7fb-4cb3-90aa-f8a537b86196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:12.723397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b5ef1-c7fb-4cb3-90aa-f8a537b86196 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:12.723671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b5ef1-c7fb-4cb3-90aa-f8a537b86196 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:12.723968 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b5ef1-c7fb-4cb3-90aa-f8a537b86196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:12.725786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-579b5ef1-c7fb-4cb3-90aa-f8a537b86196 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:12.827525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b5ef1-c7fb-4cb3-90aa-f8a537b86196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:12.829779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b5ef1-c7fb-4cb3-90aa-f8a537b86196 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:12.850287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 678/1363] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:15.129362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c843e1-2845-473c-90ba-538fc42edef1 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:15.129923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c843e1-2845-473c-90ba-538fc42edef1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:15.129923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c843e1-2845-473c-90ba-538fc42edef1 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:15.131873 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 686/1364] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:15.137287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5208e9-8939-478c-a325-ffb93659b4db None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:15.137738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5208e9-8939-478c-a325-ffb93659b4db None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:15.137847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5208e9-8939-478c-a325-ffb93659b4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:15.154982 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:15.211932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-ee5208e9-8939-478c-a325-ffb93659b4db None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:15.213138 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ee5208e9-8939-478c-a325-ffb93659b4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:15.316941 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 679/1365] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:15.619906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d731bc6-d32d-45b9-8afd-682d1ea20adb None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:15.620318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d731bc6-d32d-45b9-8afd-682d1ea20adb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:15.620591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d731bc6-d32d-45b9-8afd-682d1ea20adb None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:15.621933 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 687/1366] 10.222.0.62 () {54 vars in 962 bytes} [Thu Sep 1 16:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:15.627511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed1f65-0d6a-466a-96a7-e8b0e042dac4 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:15.627824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed1f65-0d6a-466a-96a7-e8b0e042dac4 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:15.628123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed1f65-0d6a-466a-96a7-e8b0e042dac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:15.711285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-65ed1f65-0d6a-466a-96a7-e8b0e042dac4 None None] MFA Rules not processed for user `2912c02ff6db4cbc97c80cd923238136`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:15.712439 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-65ed1f65-0d6a-466a-96a7-e8b0e042dac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:15.763433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:15.848952 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 680/1367] 10.222.0.62 () {58 vars in 1059 bytes} [Thu Sep 1 16:29:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:15.857615 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d731bc6-d32d-45b9-8afd-682d1ea20adb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:15.859242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8d731bc6-d32d-45b9-8afd-682d1ea20adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:16.020152 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d731bc6-d32d-45b9-8afd-682d1ea20adb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:16.042961 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:16.149426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939e295-741a-4946-a4f5-71138748f102 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:16.150233 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939e295-741a-4946-a4f5-71138748f102 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:16.150604 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939e295-741a-4946-a4f5-71138748f102 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:16.151100 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939e295-741a-4946-a4f5-71138748f102 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:16.152926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0939e295-741a-4946-a4f5-71138748f102 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:16.257547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939e295-741a-4946-a4f5-71138748f102 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:16.259415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939e295-741a-4946-a4f5-71138748f102 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:16.290402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 688/1368] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:16.925265 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7541c-7573-4503-8179-9f050931ada1 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:16.925833 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7541c-7573-4503-8179-9f050931ada1 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:16.926085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7541c-7573-4503-8179-9f050931ada1 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:16.927740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 681/1369] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:29:16.933119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21403b7e-bfef-40f0-a2d5-f4027c9829de None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:16.933405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21403b7e-bfef-40f0-a2d5-f4027c9829de None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:16.933677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21403b7e-bfef-40f0-a2d5-f4027c9829de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:16.967577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:17.006436 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-21403b7e-bfef-40f0-a2d5-f4027c9829de None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:17.007659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-21403b7e-bfef-40f0-a2d5-f4027c9829de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:17.109796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 689/1370] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:17.694981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4cce6-08eb-42af-9b39-9287ebb1a99d None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:17.695180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4cce6-08eb-42af-9b39-9287ebb1a99d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:17.695541 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4cce6-08eb-42af-9b39-9287ebb1a99d None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:17.697274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 682/1371] 10.222.0.62 () {54 vars in 964 bytes} [Thu Sep 1 16:29:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:17.702278 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ada75-6632-4978-9a54-d7982b23fe8c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:17.703173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ada75-6632-4978-9a54-d7982b23fe8c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:17.703173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ada75-6632-4978-9a54-d7982b23fe8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:17.796240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6c4ada75-6632-4978-9a54-d7982b23fe8c None None] MFA Rules not processed for user `6160b373ace44ccfaf557b557343bf32`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:17.797467 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6c4ada75-6632-4978-9a54-d7982b23fe8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:17.845115 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:17.877068 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 690/1372] 10.222.0.62 () {58 vars in 1061 bytes} [Thu Sep 1 16:29:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:17.885220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4cce6-08eb-42af-9b39-9287ebb1a99d None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:17.886766 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-80d4cce6-08eb-42af-9b39-9287ebb1a99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:17.999735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4cce6-08eb-42af-9b39-9287ebb1a99d None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:18.041050 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:18.080734 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ab280-e77f-4f2a-9f05-8abab525a447 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:18.081252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ab280-e77f-4f2a-9f05-8abab525a447 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:18.081616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ab280-e77f-4f2a-9f05-8abab525a447 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:18.081771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ab280-e77f-4f2a-9f05-8abab525a447 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:18.083513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f11ab280-e77f-4f2a-9f05-8abab525a447 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:18.208060 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ab280-e77f-4f2a-9f05-8abab525a447 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:18.209759 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ab280-e77f-4f2a-9f05-8abab525a447 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:18.230304 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 683/1373] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:18.294338 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7773afd-ce6f-40e4-9484-de21836a79a7 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:18.294773 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7773afd-ce6f-40e4-9484-de21836a79a7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:18.295214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7773afd-ce6f-40e4-9484-de21836a79a7 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:18.297569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 691/1374] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:29:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:18.302595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dc764-6f3d-4030-82a2-359d7553406d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:18.302937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dc764-6f3d-4030-82a2-359d7553406d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:18.303147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dc764-6f3d-4030-82a2-359d7553406d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:18.384337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-505dc764-6f3d-4030-82a2-359d7553406d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:18.385971 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-505dc764-6f3d-4030-82a2-359d7553406d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:18.413863 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:18.496054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 684/1375] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:18.785642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7773afd-ce6f-40e4-9484-de21836a79a7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:18.787169 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c7773afd-ce6f-40e4-9484-de21836a79a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:18.937600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7773afd-ce6f-40e4-9484-de21836a79a7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:18.990958 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:19.054573 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614658c-7406-45b2-ae13-9351400206cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:19.055090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614658c-7406-45b2-ae13-9351400206cc service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:19.055366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614658c-7406-45b2-ae13-9351400206cc service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:19.055631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614658c-7406-45b2-ae13-9351400206cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:19.057277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7614658c-7406-45b2-ae13-9351400206cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:19.147581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614658c-7406-45b2-ae13-9351400206cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:19.149214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614658c-7406-45b2-ae13-9351400206cc service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:19.167712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 692/1376] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:21.102072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47109ba6-cdc5-4513-81b3-0c8eefa523f5 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:21.102639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47109ba6-cdc5-4513-81b3-0c8eefa523f5 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:21.102639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47109ba6-cdc5-4513-81b3-0c8eefa523f5 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:21.103952 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 685/1377] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:21.109320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37434f47-b7c7-47f0-8d0d-622f624f8b23 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:21.109744 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37434f47-b7c7-47f0-8d0d-622f624f8b23 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:21.109856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37434f47-b7c7-47f0-8d0d-622f624f8b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:21.185321 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-37434f47-b7c7-47f0-8d0d-622f624f8b23 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:21.186535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-37434f47-b7c7-47f0-8d0d-622f624f8b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:21.193334 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:21.311479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 693/1378] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:21.607965 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47109ba6-cdc5-4513-81b3-0c8eefa523f5 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:21.607965 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-47109ba6-cdc5-4513-81b3-0c8eefa523f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:21.827499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47109ba6-cdc5-4513-81b3-0c8eefa523f5 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:21.837847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:21.957135 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44764d-c9dd-4e98-8ad8-5af8ccde753e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:21.957851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44764d-c9dd-4e98-8ad8-5af8ccde753e service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:21.958226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44764d-c9dd-4e98-8ad8-5af8ccde753e service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:21.958628 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44764d-c9dd-4e98-8ad8-5af8ccde753e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:21.960385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-4b44764d-c9dd-4e98-8ad8-5af8ccde753e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:22.054648 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44764d-c9dd-4e98-8ad8-5af8ccde753e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:22.056453 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44764d-c9dd-4e98-8ad8-5af8ccde753e service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:22.075326 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 686/1379] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:24.054747 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759754eb-9013-4d0e-be25-3df9445f57ad None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:24.055360 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759754eb-9013-4d0e-be25-3df9445f57ad None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:24.055360 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759754eb-9013-4d0e-be25-3df9445f57ad None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:24.056842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 694/1380] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:24.062174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c28fb-2d5f-4921-96e9-c00faa9e87df None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:24.062589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c28fb-2d5f-4921-96e9-c00faa9e87df None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:24.062823 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c28fb-2d5f-4921-96e9-c00faa9e87df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:24.074731 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:24.135151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e25c28fb-2d5f-4921-96e9-c00faa9e87df None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:24.136393 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e25c28fb-2d5f-4921-96e9-c00faa9e87df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:24.240142 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 687/1381] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:24.527912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759754eb-9013-4d0e-be25-3df9445f57ad None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:24.529553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-759754eb-9013-4d0e-be25-3df9445f57ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:24.695730 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:24.709682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759754eb-9013-4d0e-be25-3df9445f57ad None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:24.839727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d7fae-23b3-4541-9a03-e919fc038ece service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:24.840420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d7fae-23b3-4541-9a03-e919fc038ece service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:24.840702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d7fae-23b3-4541-9a03-e919fc038ece service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:24.841244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d7fae-23b3-4541-9a03-e919fc038ece service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:24.843192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-332d7fae-23b3-4541-9a03-e919fc038ece service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:24.937554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d7fae-23b3-4541-9a03-e919fc038ece service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:24.939160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d7fae-23b3-4541-9a03-e919fc038ece service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:24.950060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:24.959571 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 695/1382] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:27.014525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbded929-5fb7-404a-be65-22f941eb85c2 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:27.015059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbded929-5fb7-404a-be65-22f941eb85c2 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:27.015059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbded929-5fb7-404a-be65-22f941eb85c2 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:27.016450 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 688/1383] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:27.023227 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cebd74-d19e-469f-b922-722876ec46bf None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:27.023708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cebd74-d19e-469f-b922-722876ec46bf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:27.023708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cebd74-d19e-469f-b922-722876ec46bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:27.107091 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a3cebd74-d19e-469f-b922-722876ec46bf None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:27.108768 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a3cebd74-d19e-469f-b922-722876ec46bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:27.226867 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 696/1384] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:27.515638 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbded929-5fb7-404a-be65-22f941eb85c2 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:27.517225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-bbded929-5fb7-404a-be65-22f941eb85c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:27.607795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:27.689325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbded929-5fb7-404a-be65-22f941eb85c2 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:27.804788 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c435fe-40b0-4f16-92e1-c273ce1bdb40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:27.805441 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c435fe-40b0-4f16-92e1-c273ce1bdb40 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:27.805704 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c435fe-40b0-4f16-92e1-c273ce1bdb40 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:27.805961 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c435fe-40b0-4f16-92e1-c273ce1bdb40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:27.807588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-72c435fe-40b0-4f16-92e1-c273ce1bdb40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:27.859530 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:27.901232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c435fe-40b0-4f16-92e1-c273ce1bdb40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:27.902876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c435fe-40b0-4f16-92e1-c273ce1bdb40 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:27.925241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 689/1385] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:30.116740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d26ae8-6352-4a38-8087-d473956bcd42 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:30.117312 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d26ae8-6352-4a38-8087-d473956bcd42 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:30.117312 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d26ae8-6352-4a38-8087-d473956bcd42 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:30.118594 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 697/1386] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:30.124775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034800c-3493-4bcd-a849-b160ad9b677d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:30.125220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034800c-3493-4bcd-a849-b160ad9b677d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:30.125443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034800c-3493-4bcd-a849-b160ad9b677d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:30.209789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-a034800c-3493-4bcd-a849-b160ad9b677d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:30.211316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a034800c-3493-4bcd-a849-b160ad9b677d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:30.332497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 690/1387] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:30.692075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d26ae8-6352-4a38-8087-d473956bcd42 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:30.693692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-87d26ae8-6352-4a38-8087-d473956bcd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:30.780693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:30.855601 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d26ae8-6352-4a38-8087-d473956bcd42 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:30.976094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b6aa7-d1fa-4d74-baec-4099909af954 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:30.976717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b6aa7-d1fa-4d74-baec-4099909af954 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:30.976989 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b6aa7-d1fa-4d74-baec-4099909af954 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:30.977324 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b6aa7-d1fa-4d74-baec-4099909af954 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:30.978878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-828b6aa7-d1fa-4d74-baec-4099909af954 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:31.019458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:31.075580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b6aa7-d1fa-4d74-baec-4099909af954 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:31.077244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b6aa7-d1fa-4d74-baec-4099909af954 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:31.096343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 698/1388] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:33.120382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab993ab-4ec8-4dc1-9508-9f0b4821cc5d None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:33.120990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab993ab-4ec8-4dc1-9508-9f0b4821cc5d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:33.120990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab993ab-4ec8-4dc1-9508-9f0b4821cc5d None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:33.122427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 691/1389] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:33.127768 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cf724-d3f2-4ba2-a975-51f83abc6d6a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:33.128224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cf724-d3f2-4ba2-a975-51f83abc6d6a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:33.128372 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cf724-d3f2-4ba2-a975-51f83abc6d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:33.200263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-496cf724-d3f2-4ba2-a975-51f83abc6d6a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:33.201498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-496cf724-d3f2-4ba2-a975-51f83abc6d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:33.308734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 699/1390] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:33.403069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac919a5-3157-48e5-b6df-d5da742bc93c None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:33.403267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac919a5-3157-48e5-b6df-d5da742bc93c None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:33.403549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac919a5-3157-48e5-b6df-d5da742bc93c None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:33.404997 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 692/1391] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:33.410482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066f902-604d-4ebb-aac7-b5d7d3d4e23d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:33.410765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066f902-604d-4ebb-aac7-b5d7d3d4e23d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:33.411152 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066f902-604d-4ebb-aac7-b5d7d3d4e23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:33.427422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:33.478385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3066f902-604d-4ebb-aac7-b5d7d3d4e23d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:33.479607 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3066f902-604d-4ebb-aac7-b5d7d3d4e23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:33.584592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 700/1392] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:33.596271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac919a5-3157-48e5-b6df-d5da742bc93c None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:33.597860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-4ac919a5-3157-48e5-b6df-d5da742bc93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:33.630904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:33.764352 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac919a5-3157-48e5-b6df-d5da742bc93c None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:33.869847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:33.882811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb52e0-4f65-493a-8e19-6d369127fa14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:33.883453 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb52e0-4f65-493a-8e19-6d369127fa14 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:33.883724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb52e0-4f65-493a-8e19-6d369127fa14 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:33.884026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb52e0-4f65-493a-8e19-6d369127fa14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:33.885618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8edb52e0-4f65-493a-8e19-6d369127fa14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:33.886474 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca03eb-4c6f-4ebb-bad7-ec79c2680804 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:33.886763 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca03eb-4c6f-4ebb-bad7-ec79c2680804 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:33.887031 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca03eb-4c6f-4ebb-bad7-ec79c2680804 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:33.888398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 701/1393] 10.222.0.62 () {54 vars in 967 bytes} [Thu Sep 1 16:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:29:33.892931 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e850c-d611-43bb-89fc-fc0b880a7194 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:33.893192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e850c-d611-43bb-89fc-fc0b880a7194 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:33.893460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e850c-d611-43bb-89fc-fc0b880a7194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:33.972861 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-135e850c-d611-43bb-89fc-fc0b880a7194 None None] MFA Rules not processed for user `3cfe36a14a11493397e4018b773d3cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:33.974101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-135e850c-d611-43bb-89fc-fc0b880a7194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:33.980283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb52e0-4f65-493a-8e19-6d369127fa14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:33.981906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb52e0-4f65-493a-8e19-6d369127fa14 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:34.000674 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 693/1394] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:34.009439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:34.053968 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 702/1395] 10.222.0.62 () {58 vars in 1064 bytes} [Thu Sep 1 16:29:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:29:34.062128 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb52e0-4f65-493a-8e19-6d369127fa14 service nova] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:34.063332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8edb52e0-4f65-493a-8e19-6d369127fa14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:34.149439 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb52e0-4f65-493a-8e19-6d369127fa14 service nova] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:34.195832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:34.232896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121adeb3-472d-4627-af50-ac038531c0e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfe36a14a11493397e4018b773d3cc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:34.233522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121adeb3-472d-4627-af50-ac038531c0e1 service glance] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:34.233796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121adeb3-472d-4627-af50-ac038531c0e1 service glance] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:34.234050 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121adeb3-472d-4627-af50-ac038531c0e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:34.235676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-121adeb3-472d-4627-af50-ac038531c0e1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:34.352560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121adeb3-472d-4627-af50-ac038531c0e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:34.354058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121adeb3-472d-4627-af50-ac038531c0e1 service glance] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:34.373029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 694/1396] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:35.124293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479e8a6-e312-4565-90e4-39e8f3d08462 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:35.124840 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479e8a6-e312-4565-90e4-39e8f3d08462 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:35.124840 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479e8a6-e312-4565-90e4-39e8f3d08462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:35.195222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1479e8a6-e312-4565-90e4-39e8f3d08462 None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:35.196472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1479e8a6-e312-4565-90e4-39e8f3d08462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:35.286254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 703/1397] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:29:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:35.329432 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898215a2-042b-40dc-8fa8-982972ed7006 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:35.329911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898215a2-042b-40dc-8fa8-982972ed7006 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:35.330001 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898215a2-042b-40dc-8fa8-982972ed7006 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:35.331352 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 695/1398] 10.222.0.62 () {54 vars in 943 bytes} [Thu Sep 1 16:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:35.337780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66149b2f-d4c5-40ed-88c0-c4bdda93b37a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:35.338133 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66149b2f-d4c5-40ed-88c0-c4bdda93b37a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:35.338505 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66149b2f-d4c5-40ed-88c0-c4bdda93b37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:35.400560 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:35.421297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-66149b2f-d4c5-40ed-88c0-c4bdda93b37a None None] MFA Rules not processed for user `56f9bc7262b64bbab0f670576c6bdfdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:35.422581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-66149b2f-d4c5-40ed-88c0-c4bdda93b37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:35.532569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 704/1399] 10.222.0.62 () {58 vars in 1040 bytes} [Thu Sep 1 16:29:35 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:29:35.542382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898215a2-042b-40dc-8fa8-982972ed7006 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:35.544020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-898215a2-042b-40dc-8fa8-982972ed7006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:35.624239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:35.688549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898215a2-042b-40dc-8fa8-982972ed7006 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:35.797135 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71335891-ce10-4ba9-ae9c-1a47a5f25fcc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:35.797729 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71335891-ce10-4ba9-ae9c-1a47a5f25fcc service swift] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:35.798001 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71335891-ce10-4ba9-ae9c-1a47a5f25fcc service swift] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:35.798266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71335891-ce10-4ba9-ae9c-1a47a5f25fcc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:35.800011 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-71335891-ce10-4ba9-ae9c-1a47a5f25fcc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:35.860787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:35.887798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71335891-ce10-4ba9-ae9c-1a47a5f25fcc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:35.889409 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71335891-ce10-4ba9-ae9c-1a47a5f25fcc service swift] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:35.897660 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 696/1400] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:29:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:29:35.963591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef77c-0ab2-4b01-9130-2247bcfdd059 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:35.963922 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef77c-0ab2-4b01-9130-2247bcfdd059 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:35.964231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef77c-0ab2-4b01-9130-2247bcfdd059 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:35.965653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 705/1401] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:35.971868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71065d14-47a8-4524-9eae-ba68e9bee1f3 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:35.972204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71065d14-47a8-4524-9eae-ba68e9bee1f3 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:35.972477 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71065d14-47a8-4524-9eae-ba68e9bee1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:36.054200 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-71065d14-47a8-4524-9eae-ba68e9bee1f3 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:36.055524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-71065d14-47a8-4524-9eae-ba68e9bee1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:36.163232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 697/1402] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:36.455039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ef77c-0ab2-4b01-9130-2247bcfdd059 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:36.455586 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ec3ef77c-0ab2-4b01-9130-2247bcfdd059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:36.588632 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:36.651802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ef77c-0ab2-4b01-9130-2247bcfdd059 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:36.794551 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc38a95-8a5c-4697-ba93-c7cec49e3380 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:36.795199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc38a95-8a5c-4697-ba93-c7cec49e3380 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:36.795586 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc38a95-8a5c-4697-ba93-c7cec49e3380 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:36.795860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc38a95-8a5c-4697-ba93-c7cec49e3380 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:36.797652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9fc38a95-8a5c-4697-ba93-c7cec49e3380 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:36.876980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:36.897461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc38a95-8a5c-4697-ba93-c7cec49e3380 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:36.899079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc38a95-8a5c-4697-ba93-c7cec49e3380 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:36.918363 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 706/1403] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:38.855407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8486c7-2c74-4baf-ab3b-f337df06edde None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:38.856157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8486c7-2c74-4baf-ab3b-f337df06edde None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:38.856157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8486c7-2c74-4baf-ab3b-f337df06edde None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:38.857329 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 698/1404] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:38.862765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08ed17-945e-45d0-bfa1-0e011d3a1131 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:38.863206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08ed17-945e-45d0-bfa1-0e011d3a1131 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:38.863296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08ed17-945e-45d0-bfa1-0e011d3a1131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:38.937506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-db08ed17-945e-45d0-bfa1-0e011d3a1131 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:38.938846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-db08ed17-945e-45d0-bfa1-0e011d3a1131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:39.044435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 707/1405] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:39.340391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8486c7-2c74-4baf-ab3b-f337df06edde None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:39.341971 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-bc8486c7-2c74-4baf-ab3b-f337df06edde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:39.410448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:39.578542 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8486c7-2c74-4baf-ab3b-f337df06edde None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:39.721082 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd47cbf-e968-4a86-a0cc-2b807d77f4b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:39.722504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd47cbf-e968-4a86-a0cc-2b807d77f4b6 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:39.722781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd47cbf-e968-4a86-a0cc-2b807d77f4b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:39.723046 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd47cbf-e968-4a86-a0cc-2b807d77f4b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:39.725195 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8fd47cbf-e968-4a86-a0cc-2b807d77f4b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:39.737241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:39.829919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd47cbf-e968-4a86-a0cc-2b807d77f4b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:39.831456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd47cbf-e968-4a86-a0cc-2b807d77f4b6 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:39.856087 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 699/1406] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:41.893494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa0dd2-f5d9-4a52-8934-fbbce4aff996 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:41.894032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa0dd2-f5d9-4a52-8934-fbbce4aff996 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:41.894032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa0dd2-f5d9-4a52-8934-fbbce4aff996 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:41.896103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 708/1407] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:41.901273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ca0a1-ba0e-447c-a6d7-73307f9a2c62 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:41.901709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ca0a1-ba0e-447c-a6d7-73307f9a2c62 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:41.901788 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ca0a1-ba0e-447c-a6d7-73307f9a2c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:41.980545 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-c78ca0a1-ba0e-447c-a6d7-73307f9a2c62 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:41.981786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c78ca0a1-ba0e-447c-a6d7-73307f9a2c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:42.079746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:42.099594 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 700/1408] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:42.388318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa0dd2-f5d9-4a52-8934-fbbce4aff996 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:42.390112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b2aa0dd2-f5d9-4a52-8934-fbbce4aff996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:42.447353 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:42.555582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa0dd2-f5d9-4a52-8934-fbbce4aff996 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:42.675872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0178e7f-92a0-4122-baab-53999f751e32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:42.676530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0178e7f-92a0-4122-baab-53999f751e32 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:42.676827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0178e7f-92a0-4122-baab-53999f751e32 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:42.677072 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0178e7f-92a0-4122-baab-53999f751e32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:42.678732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b0178e7f-92a0-4122-baab-53999f751e32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:42.688282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:42.779135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0178e7f-92a0-4122-baab-53999f751e32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:42.781118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0178e7f-92a0-4122-baab-53999f751e32 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:42.808071 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 709/1409] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:52.096347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=5 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:52.798809 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=41 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:58.138843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90094013-8890-4787-89f0-e7d6642950b3 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:58.139479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90094013-8890-4787-89f0-e7d6642950b3 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:58.139479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90094013-8890-4787-89f0-e7d6642950b3 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:58.140785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 701/1410] 10.222.0.62 () {54 vars in 918 bytes} [Thu Sep 1 16:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:58.146885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc657d4-c886-4b8a-9c3d-09dd58cf84cd None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:58.147369 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc657d4-c886-4b8a-9c3d-09dd58cf84cd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:58.147557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc657d4-c886-4b8a-9c3d-09dd58cf84cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:58.355420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dbc657d4-c886-4b8a-9c3d-09dd58cf84cd None None] MFA Rules not processed for user `6281dcf2927a4ed2870125df8801be78`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:58.356691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dbc657d4-c886-4b8a-9c3d-09dd58cf84cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:58.472548 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 710/1411] 10.222.0.62 () {58 vars in 1015 bytes} [Thu Sep 1 16:29:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:58.487939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c5336-b308-435d-87bb-8efd60738690 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:58.488201 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c5336-b308-435d-87bb-8efd60738690 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:58.488495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c5336-b308-435d-87bb-8efd60738690 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:58.489841 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 702/1412] 10.222.0.62 () {54 vars in 918 bytes} [Thu Sep 1 16:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:58.495250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fba79-4315-4d6a-a50c-58638d377332 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:58.495529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fba79-4315-4d6a-a50c-58638d377332 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:58.495791 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fba79-4315-4d6a-a50c-58638d377332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:58.580094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:58.582786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-982fba79-4315-4d6a-a50c-58638d377332 None None] MFA Rules not processed for user `2912c02ff6db4cbc97c80cd923238136`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:58.584822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-982fba79-4315-4d6a-a50c-58638d377332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:58.712226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 711/1413] 10.222.0.62 () {58 vars in 1015 bytes} [Thu Sep 1 16:29:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:01.089591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf30cd-b68b-4e61-8741-5d4ea64618a0 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:01.090533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf30cd-b68b-4e61-8741-5d4ea64618a0 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:01.090533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf30cd-b68b-4e61-8741-5d4ea64618a0 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:01.092722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:30:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:01.099252 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c09490-493a-49ba-bfd9-a1e367647cca None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:01.100292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c09490-493a-49ba-bfd9-a1e367647cca None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:01.100565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c09490-493a-49ba-bfd9-a1e367647cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:01.188408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-68c09490-493a-49ba-bfd9-a1e367647cca None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:01.190219 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-68c09490-493a-49ba-bfd9-a1e367647cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:01.252262 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:01.304453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:01.594820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf30cd-b68b-4e61-8741-5d4ea64618a0 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:01.596383 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-dedf30cd-b68b-4e61-8741-5d4ea64618a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:01.756592 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf30cd-b68b-4e61-8741-5d4ea64618a0 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:01.834052 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:01.878136 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eaa369-0863-4df5-95c8-91fc3a54ff80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:01.878729 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaa369-0863-4df5-95c8-91fc3a54ff80 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:01.879020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaa369-0863-4df5-95c8-91fc3a54ff80 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:01.879284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaa369-0863-4df5-95c8-91fc3a54ff80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:01.881006 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f8eaa369-0863-4df5-95c8-91fc3a54ff80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:01.997128 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eaa369-0863-4df5-95c8-91fc3a54ff80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:01.998791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eaa369-0863-4df5-95c8-91fc3a54ff80 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:02.018146 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 704/1416] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:02.927228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4811245-e60c-4226-9fe6-ded59f2320a9 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:02.928599 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4811245-e60c-4226-9fe6-ded59f2320a9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:02.928599 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4811245-e60c-4226-9fe6-ded59f2320a9 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:02.929599 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:30:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:02.934809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3306984-138b-4245-a4f5-e75d07c7f93d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:02.935234 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3306984-138b-4245-a4f5-e75d07c7f93d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:02.935346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3306984-138b-4245-a4f5-e75d07c7f93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:03.017071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-f3306984-138b-4245-a4f5-e75d07c7f93d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:03.018170 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f3306984-138b-4245-a4f5-e75d07c7f93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:03.045578 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:03.118349 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:03.270252 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4811245-e60c-4226-9fe6-ded59f2320a9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:03.271814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f4811245-e60c-4226-9fe6-ded59f2320a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:03.432201 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4811245-e60c-4226-9fe6-ded59f2320a9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:03.481857 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:03.566531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e222ba-51ce-4caa-b0b7-f3976f5b7c20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:03.567156 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e222ba-51ce-4caa-b0b7-f3976f5b7c20 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:03.567426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e222ba-51ce-4caa-b0b7-f3976f5b7c20 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:03.567687 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e222ba-51ce-4caa-b0b7-f3976f5b7c20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:03.569525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-56e222ba-51ce-4caa-b0b7-f3976f5b7c20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:03.665330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e222ba-51ce-4caa-b0b7-f3976f5b7c20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:03.666968 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e222ba-51ce-4caa-b0b7-f3976f5b7c20 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:03.687256 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 714/1419] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:04.584901 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017e098-ea42-4d1e-a41d-055e53a1db31 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:04.585830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017e098-ea42-4d1e-a41d-055e53a1db31 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:04.585830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017e098-ea42-4d1e-a41d-055e53a1db31 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:04.587945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:30:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:04.596113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c7ca1-04f8-4461-bf77-4160eaa22a82 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:04.596793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c7ca1-04f8-4461-bf77-4160eaa22a82 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:04.596986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c7ca1-04f8-4461-bf77-4160eaa22a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:04.662480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:04.687139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-606c7ca1-04f8-4461-bf77-4160eaa22a82 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:04.688411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-606c7ca1-04f8-4461-bf77-4160eaa22a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:04.790974 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:04.943982 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017e098-ea42-4d1e-a41d-055e53a1db31 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:04.946521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e017e098-ea42-4d1e-a41d-055e53a1db31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:05.108588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017e098-ea42-4d1e-a41d-055e53a1db31 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:05.121225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:05.237555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d751829-b24c-42b3-9d1a-8e0d4e0530a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:05.238089 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d751829-b24c-42b3-9d1a-8e0d4e0530a6 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:05.238369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d751829-b24c-42b3-9d1a-8e0d4e0530a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:05.238635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d751829-b24c-42b3-9d1a-8e0d4e0530a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:05.240368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6d751829-b24c-42b3-9d1a-8e0d4e0530a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:05.345464 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d751829-b24c-42b3-9d1a-8e0d4e0530a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:05.347068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d751829-b24c-42b3-9d1a-8e0d4e0530a6 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:05.367281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 707/1422] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:06.264317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99ef5c-e777-40a8-a74f-8cc534b404cf None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:06.265235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99ef5c-e777-40a8-a74f-8cc534b404cf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:06.265235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99ef5c-e777-40a8-a74f-8cc534b404cf None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:06.267356 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:30:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:06.273487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097549f6-71da-4227-9235-9e9604199937 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:06.274213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097549f6-71da-4227-9235-9e9604199937 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:06.274455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097549f6-71da-4227-9235-9e9604199937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:06.299871 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:06.356395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-097549f6-71da-4227-9235-9e9604199937 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:06.357594 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-097549f6-71da-4227-9235-9e9604199937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:06.465253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:06.625814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99ef5c-e777-40a8-a74f-8cc534b404cf None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:06.628261 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7e99ef5c-e777-40a8-a74f-8cc534b404cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:06.756581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:06.796218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99ef5c-e777-40a8-a74f-8cc534b404cf None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:06.926502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d61f08-faa1-4fe5-b71e-36c2c30677a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:06.927027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61f08-faa1-4fe5-b71e-36c2c30677a7 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:06.927301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61f08-faa1-4fe5-b71e-36c2c30677a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:06.927605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61f08-faa1-4fe5-b71e-36c2c30677a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:06.929593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-31d61f08-faa1-4fe5-b71e-36c2c30677a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:07.010919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:07.026493 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d61f08-faa1-4fe5-b71e-36c2c30677a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:07.028259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d61f08-faa1-4fe5-b71e-36c2c30677a7 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:07.047713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 717/1425] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:07.903623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1720e86-24ba-44a0-93c4-fe133796ac48 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:07.904573 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1720e86-24ba-44a0-93c4-fe133796ac48 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:07.904573 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1720e86-24ba-44a0-93c4-fe133796ac48 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:07.906897 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:30:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:07.914979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33834e-eca5-4b43-920d-3e0a78bc24fb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:07.915434 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33834e-eca5-4b43-920d-3e0a78bc24fb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:07.915539 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33834e-eca5-4b43-920d-3e0a78bc24fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:08.027839 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5b33834e-eca5-4b43-920d-3e0a78bc24fb None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:08.029004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5b33834e-eca5-4b43-920d-3e0a78bc24fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:08.131553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:08.293106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1720e86-24ba-44a0-93c4-fe133796ac48 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:08.295387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f1720e86-24ba-44a0-93c4-fe133796ac48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:08.385816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:08.456924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1720e86-24ba-44a0-93c4-fe133796ac48 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:08.590568 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47794f-95cb-4242-869e-ab08c33e55c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:08.591182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47794f-95cb-4242-869e-ab08c33e55c6 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:08.591454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47794f-95cb-4242-869e-ab08c33e55c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:08.591712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47794f-95cb-4242-869e-ab08c33e55c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:08.593361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ce47794f-95cb-4242-869e-ab08c33e55c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:08.644313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:08.687012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47794f-95cb-4242-869e-ab08c33e55c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:08.688700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47794f-95cb-4242-869e-ab08c33e55c6 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:08.707707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 710/1428] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:09.506905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25943-3f4d-47c6-b3a9-9c39a701b6b2 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:09.507474 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25943-3f4d-47c6-b3a9-9c39a701b6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:09.507474 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d25943-3f4d-47c6-b3a9-9c39a701b6b2 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:09.508712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:09.514635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e8ca3-9ca6-44b4-828a-cf4833bb12f4 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:09.515326 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e8ca3-9ca6-44b4-828a-cf4833bb12f4 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:09.515681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e8ca3-9ca6-44b4-828a-cf4833bb12f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:09.604507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-971e8ca3-9ca6-44b4-828a-cf4833bb12f4 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:09.605725 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-971e8ca3-9ca6-44b4-828a-cf4833bb12f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:09.719984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:30:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:09.879284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d25943-3f4d-47c6-b3a9-9c39a701b6b2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:09.881940 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e2d25943-3f4d-47c6-b3a9-9c39a701b6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:09.935558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:10.048008 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d25943-3f4d-47c6-b3a9-9c39a701b6b2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:10.163941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a53856-52d4-484a-afd1-8673f7526cbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:10.164569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a53856-52d4-484a-afd1-8673f7526cbf service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:10.164837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a53856-52d4-484a-afd1-8673f7526cbf service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:10.165100 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a53856-52d4-484a-afd1-8673f7526cbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:10.166685 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-19a53856-52d4-484a-afd1-8673f7526cbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:10.174215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:10.273671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a53856-52d4-484a-afd1-8673f7526cbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:10.275275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a53856-52d4-484a-afd1-8673f7526cbf service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:10.294872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 720/1431] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:11.129838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3874e-359d-44a6-910d-eca95d8285b4 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:11.130415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3874e-359d-44a6-910d-eca95d8285b4 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:11.130415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3874e-359d-44a6-910d-eca95d8285b4 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:11.132063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:11.137431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a4e84-a834-44c5-bbde-ba4d0cd9bbcb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:11.137958 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a4e84-a834-44c5-bbde-ba4d0cd9bbcb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:11.138082 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a4e84-a834-44c5-bbde-ba4d0cd9bbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:11.219532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-961a4e84-a834-44c5-bbde-ba4d0cd9bbcb None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:11.220708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-961a4e84-a834-44c5-bbde-ba4d0cd9bbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:11.291317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:11.333239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:11.508403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3874e-359d-44a6-910d-eca95d8285b4 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:11.510530 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-26a3874e-359d-44a6-910d-eca95d8285b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:11.539267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:11.678407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3874e-359d-44a6-910d-eca95d8285b4 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:11.779961 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:11.806578 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774d428-8d48-455f-9f99-46aa51c3ef81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:11.807200 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774d428-8d48-455f-9f99-46aa51c3ef81 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:11.807475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774d428-8d48-455f-9f99-46aa51c3ef81 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:11.807736 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774d428-8d48-455f-9f99-46aa51c3ef81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:11.809397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8774d428-8d48-455f-9f99-46aa51c3ef81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:11.904016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774d428-8d48-455f-9f99-46aa51c3ef81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:11.905641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774d428-8d48-455f-9f99-46aa51c3ef81 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:11.925055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 713/1434] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:12.746161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8627751-beaf-42c1-9112-f72337773134 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:12.747064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8627751-beaf-42c1-9112-f72337773134 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:12.747064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8627751-beaf-42c1-9112-f72337773134 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:12.748844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:30:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:12.754449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f601ac-cf3b-4ae8-ab29-524cde488591 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:12.755164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f601ac-cf3b-4ae8-ab29-524cde488591 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:12.755330 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f601ac-cf3b-4ae8-ab29-524cde488591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:12.838656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-08f601ac-cf3b-4ae8-ab29-524cde488591 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:12.839870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-08f601ac-cf3b-4ae8-ab29-524cde488591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:12.884013 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:12.956210 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:13.104712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8627751-beaf-42c1-9112-f72337773134 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:13.106388 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b8627751-beaf-42c1-9112-f72337773134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:13.280292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8627751-beaf-42c1-9112-f72337773134 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:13.343668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:13.402648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7f75b-1ea3-47ab-ab19-f7830f4e2da3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:13.403177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7f75b-1ea3-47ab-ab19-f7830f4e2da3 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:13.403442 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7f75b-1ea3-47ab-ab19-f7830f4e2da3 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:13.403707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7f75b-1ea3-47ab-ab19-f7830f4e2da3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:13.405322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-35d7f75b-1ea3-47ab-ab19-f7830f4e2da3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:13.501748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7f75b-1ea3-47ab-ab19-f7830f4e2da3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:13.503410 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7f75b-1ea3-47ab-ab19-f7830f4e2da3 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:13.522283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 723/1437] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:14.341066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003044d-54bf-4943-855b-3f1d9dd856da None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:14.341711 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003044d-54bf-4943-855b-3f1d9dd856da None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:14.341819 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003044d-54bf-4943-855b-3f1d9dd856da None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:14.343187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:14.349362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b6fde-31b0-465e-a088-5af0bb4a72a3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:14.350078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b6fde-31b0-465e-a088-5af0bb4a72a3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:14.350240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b6fde-31b0-465e-a088-5af0bb4a72a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:14.434359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a45b6fde-31b0-465e-a088-5af0bb4a72a3 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:14.435574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a45b6fde-31b0-465e-a088-5af0bb4a72a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:14.437850 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:14.535415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:14.684878 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003044d-54bf-4943-855b-3f1d9dd856da None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:14.686383 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1003044d-54bf-4943-855b-3f1d9dd856da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:14.859641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003044d-54bf-4943-855b-3f1d9dd856da None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:14.888407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:14.984848 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b1388-a16b-45ae-b398-0c4134600cc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:14.985382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b1388-a16b-45ae-b398-0c4134600cc6 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:14.985657 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b1388-a16b-45ae-b398-0c4134600cc6 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:14.985923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b1388-a16b-45ae-b398-0c4134600cc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:14.987557 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b23b1388-a16b-45ae-b398-0c4134600cc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:15.086877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b1388-a16b-45ae-b398-0c4134600cc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:15.088570 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b1388-a16b-45ae-b398-0c4134600cc6 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:15.109111 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 716/1440] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:16.965333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1ee7c-3358-4c6b-a711-745a9a3596d0 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:16.965851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1ee7c-3358-4c6b-a711-745a9a3596d0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:16.965851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1ee7c-3358-4c6b-a711-745a9a3596d0 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:16.967118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:16.973296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da70fb1-c972-4989-bde8-65ded1848894 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:16.973742 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da70fb1-c972-4989-bde8-65ded1848894 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:16.973890 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da70fb1-c972-4989-bde8-65ded1848894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:17.025786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:17.063111 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-1da70fb1-c972-4989-bde8-65ded1848894 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:17.064381 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1da70fb1-c972-4989-bde8-65ded1848894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:17.175336 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:17.459335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1ee7c-3358-4c6b-a711-745a9a3596d0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:17.460942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-05b1ee7c-3358-4c6b-a711-745a9a3596d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:17.576964 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1ee7c-3358-4c6b-a711-745a9a3596d0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:17.616817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:17.661544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f222db-ed10-46c2-85cd-680b5d3655b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:17.662154 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f222db-ed10-46c2-85cd-680b5d3655b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:17.662424 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f222db-ed10-46c2-85cd-680b5d3655b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:17.662689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f222db-ed10-46c2-85cd-680b5d3655b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:17.664865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-74f222db-ed10-46c2-85cd-680b5d3655b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:17.777837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f222db-ed10-46c2-85cd-680b5d3655b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:17.779285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f222db-ed10-46c2-85cd-680b5d3655b7 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:17.799092 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 726/1443] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:30:19.887619 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be148b1f-cabc-44f7-9ad8-c3c2681f5818 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:19.888630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be148b1f-cabc-44f7-9ad8-c3c2681f5818 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:19.888630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be148b1f-cabc-44f7-9ad8-c3c2681f5818 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:19.890790 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:30:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:19.897128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75915157-9bf1-40e7-836f-05494e97de2e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:19.897841 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75915157-9bf1-40e7-836f-05494e97de2e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:19.898023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75915157-9bf1-40e7-836f-05494e97de2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:19.982849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-75915157-9bf1-40e7-836f-05494e97de2e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:19.984795 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-75915157-9bf1-40e7-836f-05494e97de2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:20.002208 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:20.103700 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:20.388363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be148b1f-cabc-44f7-9ad8-c3c2681f5818 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:20.389917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-be148b1f-cabc-44f7-9ad8-c3c2681f5818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:20.499484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be148b1f-cabc-44f7-9ad8-c3c2681f5818 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:20.512782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:20.577925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06535f-bff4-457c-9625-3f7b8d8059d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:20.578888 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06535f-bff4-457c-9625-3f7b8d8059d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:20.579166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06535f-bff4-457c-9625-3f7b8d8059d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:20.579427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06535f-bff4-457c-9625-3f7b8d8059d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:20.581072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1a06535f-bff4-457c-9625-3f7b8d8059d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:20.700262 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06535f-bff4-457c-9625-3f7b8d8059d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:20.701650 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06535f-bff4-457c-9625-3f7b8d8059d6 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:20.719832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 719/1446] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:22.950803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55817cf-ae54-4e33-bfd0-a9df22f958d2 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:22.951929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55817cf-ae54-4e33-bfd0-a9df22f958d2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:22.951929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55817cf-ae54-4e33-bfd0-a9df22f958d2 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:22.954381 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:30:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:22.960626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953560d6-8c76-447b-815c-cdd245c6e0a0 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:22.961371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953560d6-8c76-447b-815c-cdd245c6e0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:22.961478 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953560d6-8c76-447b-815c-cdd245c6e0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:23.017518 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:23.037971 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-953560d6-8c76-447b-815c-cdd245c6e0a0 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:23.039197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-953560d6-8c76-447b-815c-cdd245c6e0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:23.141659 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:23.424329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55817cf-ae54-4e33-bfd0-a9df22f958d2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:23.426806 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f55817cf-ae54-4e33-bfd0-a9df22f958d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:23.561215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55817cf-ae54-4e33-bfd0-a9df22f958d2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:23.604976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:23.643848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ff7eb-cead-4c4d-bf53-ab829fc7b34e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:23.644536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ff7eb-cead-4c4d-bf53-ab829fc7b34e service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:23.644800 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ff7eb-cead-4c4d-bf53-ab829fc7b34e service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:23.645060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ff7eb-cead-4c4d-bf53-ab829fc7b34e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:23.646688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a33ff7eb-cead-4c4d-bf53-ab829fc7b34e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:23.775238 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ff7eb-cead-4c4d-bf53-ab829fc7b34e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:23.776729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ff7eb-cead-4c4d-bf53-ab829fc7b34e service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:23.795536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 729/1449] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:25.901015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252de8b-fde9-431d-8953-652141731e27 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:25.901597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252de8b-fde9-431d-8953-652141731e27 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:25.901597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252de8b-fde9-431d-8953-652141731e27 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:25.902909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:25.908502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d84f6-1b83-4081-89cb-87d16012d4e1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:25.908934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d84f6-1b83-4081-89cb-87d16012d4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:25.909052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d84f6-1b83-4081-89cb-87d16012d4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:25.982414 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-750d84f6-1b83-4081-89cb-87d16012d4e1 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:25.983648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-750d84f6-1b83-4081-89cb-87d16012d4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:26.006145 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:26.089682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:26.374830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252de8b-fde9-431d-8953-652141731e27 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:26.376403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b252de8b-fde9-431d-8953-652141731e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:26.496996 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252de8b-fde9-431d-8953-652141731e27 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:26.522621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:26.581291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b8365-5722-4e45-be83-63b423693514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:26.581914 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b8365-5722-4e45-be83-63b423693514 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:26.582190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b8365-5722-4e45-be83-63b423693514 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:26.582461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b8365-5722-4e45-be83-63b423693514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:26.584124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0d7b8365-5722-4e45-be83-63b423693514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:26.700142 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b8365-5722-4e45-be83-63b423693514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:26.701513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b8365-5722-4e45-be83-63b423693514 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:26.720665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 722/1452] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:28.710101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637af2-b8c7-4452-847f-5d0bdde2f1ae None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:28.710701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637af2-b8c7-4452-847f-5d0bdde2f1ae None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:28.710701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637af2-b8c7-4452-847f-5d0bdde2f1ae None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:28.712030 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:28.718098 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aece2cb-c6bf-4186-9b82-566e7e651e1e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:28.718563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aece2cb-c6bf-4186-9b82-566e7e651e1e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:28.718659 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aece2cb-c6bf-4186-9b82-566e7e651e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:28.791825 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:28.794932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-9aece2cb-c6bf-4186-9b82-566e7e651e1e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:28.796762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9aece2cb-c6bf-4186-9b82-566e7e651e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:28.908567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:29.192343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637af2-b8c7-4452-847f-5d0bdde2f1ae None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:29.194726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b3637af2-b8c7-4452-847f-5d0bdde2f1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:29.320041 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637af2-b8c7-4452-847f-5d0bdde2f1ae None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:29.370924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:29.395909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af8103-a653-41cc-bb20-6af8e5690a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:29.396528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af8103-a653-41cc-bb20-6af8e5690a41 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:29.396787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af8103-a653-41cc-bb20-6af8e5690a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:29.397061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af8103-a653-41cc-bb20-6af8e5690a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:29.398641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-89af8103-a653-41cc-bb20-6af8e5690a41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:29.516242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af8103-a653-41cc-bb20-6af8e5690a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:29.517595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af8103-a653-41cc-bb20-6af8e5690a41 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:29.538207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 732/1455] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:31.525750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919360e1-d676-44a9-bb13-63c7216fb31e None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:31.526282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919360e1-d676-44a9-bb13-63c7216fb31e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:31.526282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919360e1-d676-44a9-bb13-63c7216fb31e None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:31.528383 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:30:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:31.533846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63daac-513f-45ac-86c8-a89a884e3dfc None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:31.534292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63daac-513f-45ac-86c8-a89a884e3dfc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:31.534399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63daac-513f-45ac-86c8-a89a884e3dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:31.609932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cb63daac-513f-45ac-86c8-a89a884e3dfc None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:31.611176 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cb63daac-513f-45ac-86c8-a89a884e3dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:31.650719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:31.722340 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:32.010213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919360e1-d676-44a9-bb13-63c7216fb31e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:32.011766 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-919360e1-d676-44a9-bb13-63c7216fb31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:32.138706 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919360e1-d676-44a9-bb13-63c7216fb31e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:32.177701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:32.227772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601143a6-5727-41a8-9522-778a3ea54589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:32.228492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601143a6-5727-41a8-9522-778a3ea54589 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:32.228767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601143a6-5727-41a8-9522-778a3ea54589 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:32.229033 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601143a6-5727-41a8-9522-778a3ea54589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:32.230639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-601143a6-5727-41a8-9522-778a3ea54589 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:32.353026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601143a6-5727-41a8-9522-778a3ea54589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:32.354479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601143a6-5727-41a8-9522-778a3ea54589 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:32.374392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 725/1458] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:34.347627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ba00a-054a-45e1-a81b-6c5d85cb8ebf None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:34.348258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ba00a-054a-45e1-a81b-6c5d85cb8ebf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:34.348258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ba00a-054a-45e1-a81b-6c5d85cb8ebf None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:34.349540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:34.355298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48fdb0-1f50-492e-894c-cfa0a80350ee None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:34.355747 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48fdb0-1f50-492e-894c-cfa0a80350ee None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:34.355838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48fdb0-1f50-492e-894c-cfa0a80350ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:34.428930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-bf48fdb0-1f50-492e-894c-cfa0a80350ee None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:34.430151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-bf48fdb0-1f50-492e-894c-cfa0a80350ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:34.439674 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:34.544360 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:34.839411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ba00a-054a-45e1-a81b-6c5d85cb8ebf None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:34.841910 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-251ba00a-054a-45e1-a81b-6c5d85cb8ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:34.954391 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ba00a-054a-45e1-a81b-6c5d85cb8ebf None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:35.022455 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:35.041909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f3c2c-042d-4f62-8c50-a58ba7e6c34c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:35.042529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f3c2c-042d-4f62-8c50-a58ba7e6c34c service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:35.042805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f3c2c-042d-4f62-8c50-a58ba7e6c34c service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:35.043069 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f3c2c-042d-4f62-8c50-a58ba7e6c34c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:35.044718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-943f3c2c-042d-4f62-8c50-a58ba7e6c34c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:35.186414 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f3c2c-042d-4f62-8c50-a58ba7e6c34c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:35.187882 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f3c2c-042d-4f62-8c50-a58ba7e6c34c service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:35.208690 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 735/1461] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:37.169176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f961ed9-3ffc-48da-ac89-a6be09717583 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:37.170031 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f961ed9-3ffc-48da-ac89-a6be09717583 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:37.170031 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f961ed9-3ffc-48da-ac89-a6be09717583 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:37.172233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:30:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:37.177968 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc736ca0-757d-4e71-98f0-4f0fb7e4efca None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:37.178392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc736ca0-757d-4e71-98f0-4f0fb7e4efca None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:37.178484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc736ca0-757d-4e71-98f0-4f0fb7e4efca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:37.245122 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dc736ca0-757d-4e71-98f0-4f0fb7e4efca None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:37.246335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dc736ca0-757d-4e71-98f0-4f0fb7e4efca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:37.295378 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:37.363583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:37.644946 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f961ed9-3ffc-48da-ac89-a6be09717583 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:37.646501 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8f961ed9-3ffc-48da-ac89-a6be09717583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:37.753465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f961ed9-3ffc-48da-ac89-a6be09717583 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:37.792932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:37.831486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6092a2-2afd-4337-a18d-0126354bb249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:37.832161 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6092a2-2afd-4337-a18d-0126354bb249 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:37.832439 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6092a2-2afd-4337-a18d-0126354bb249 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:37.832712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6092a2-2afd-4337-a18d-0126354bb249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:37.834344 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ed6092a2-2afd-4337-a18d-0126354bb249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:37.946267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6092a2-2afd-4337-a18d-0126354bb249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:37.948114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6092a2-2afd-4337-a18d-0126354bb249 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:37.967564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 728/1464] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:39.927404 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ac06d-e23e-49d0-b610-6658a49cada2 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.927985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ac06d-e23e-49d0-b610-6658a49cada2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.928075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ac06d-e23e-49d0-b610-6658a49cada2 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:39.929454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:39.935258 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57d3ff-9bea-4b3e-a5e1-9989a3fe80b8 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:39.935724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57d3ff-9bea-4b3e-a5e1-9989a3fe80b8 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:39.935840 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57d3ff-9bea-4b3e-a5e1-9989a3fe80b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.010027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-2c57d3ff-9bea-4b3e-a5e1-9989a3fe80b8 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:40.010251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2c57d3ff-9bea-4b3e-a5e1-9989a3fe80b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:40.030218 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:40.114799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:40.397962 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ac06d-e23e-49d0-b610-6658a49cada2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:40.399813 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3f3ac06d-e23e-49d0-b610-6658a49cada2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:40.515495 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ac06d-e23e-49d0-b610-6658a49cada2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:40.587801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:40.596663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a2625-b18d-496e-9e7e-eb81641398ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:40.597104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a2625-b18d-496e-9e7e-eb81641398ab service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:40.597449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a2625-b18d-496e-9e7e-eb81641398ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:40.597788 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a2625-b18d-496e-9e7e-eb81641398ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:40.599394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2b8a2625-b18d-496e-9e7e-eb81641398ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:40.716707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a2625-b18d-496e-9e7e-eb81641398ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:40.718073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a2625-b18d-496e-9e7e-eb81641398ab service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:40.736859 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 738/1467] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:42.724716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ee5a6-1ad0-4f72-8010-fbc8f2fe54fa None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:42.725313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ee5a6-1ad0-4f72-8010-fbc8f2fe54fa None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:42.725313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ee5a6-1ad0-4f72-8010-fbc8f2fe54fa None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:42.726767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:42.732785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059580c-7d94-43b9-944f-5ab9045acdfb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:42.733253 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059580c-7d94-43b9-944f-5ab9045acdfb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:42.733367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059580c-7d94-43b9-944f-5ab9045acdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:42.810206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7059580c-7d94-43b9-944f-5ab9045acdfb None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:42.811580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7059580c-7d94-43b9-944f-5ab9045acdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:42.865069 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:42.916701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:43.199387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ee5a6-1ad0-4f72-8010-fbc8f2fe54fa None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:43.199717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f39ee5a6-1ad0-4f72-8010-fbc8f2fe54fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:43.313245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ee5a6-1ad0-4f72-8010-fbc8f2fe54fa None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:43.366736 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:43.398025 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c1770-9e0e-44dc-a716-57bbba36984e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:43.398641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c1770-9e0e-44dc-a716-57bbba36984e service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:43.398914 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c1770-9e0e-44dc-a716-57bbba36984e service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:43.399209 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c1770-9e0e-44dc-a716-57bbba36984e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:43.400976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-db9c1770-9e0e-44dc-a716-57bbba36984e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:43.527078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c1770-9e0e-44dc-a716-57bbba36984e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:43.528585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c1770-9e0e-44dc-a716-57bbba36984e service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:43.548866 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 731/1470] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:52.914805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=22 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:53.546309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=66 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:03.316059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2131834-5221-4c8b-aab3-ba7d6e9df436 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:03.316627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2131834-5221-4c8b-aab3-ba7d6e9df436 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:03.316627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2131834-5221-4c8b-aab3-ba7d6e9df436 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:03.317941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 740/1471] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:03.324375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8df17b-83cb-494c-b079-2488dd99d4b0 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:03.324864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8df17b-83cb-494c-b079-2488dd99d4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:03.325085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8df17b-83cb-494c-b079-2488dd99d4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:03.395408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-ab8df17b-83cb-494c-b079-2488dd99d4b0 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:03.396660 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ab8df17b-83cb-494c-b079-2488dd99d4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:03.501092 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 732/1472] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:03.810161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190aaab8-9ca9-413e-8edf-3a3d01d5bb32 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:03.810635 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190aaab8-9ca9-413e-8edf-3a3d01d5bb32 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:03.811077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190aaab8-9ca9-413e-8edf-3a3d01d5bb32 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:03.813299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 741/1473] 10.222.0.62 () {54 vars in 967 bytes} [Thu Sep 1 16:31:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:03.821158 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7abcd8-e650-4e61-a9af-b711d978f08e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:03.821640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7abcd8-e650-4e61-a9af-b711d978f08e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:03.822092 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7abcd8-e650-4e61-a9af-b711d978f08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:03.910079 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-ab7abcd8-e650-4e61-a9af-b711d978f08e None None] MFA Rules not processed for user `3cfe36a14a11493397e4018b773d3cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:03.911432 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ab7abcd8-e650-4e61-a9af-b711d978f08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:03.934430 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:03.996225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 733/1474] 10.222.0.62 () {58 vars in 1064 bytes} [Thu Sep 1 16:31:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:04.004299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190aaab8-9ca9-413e-8edf-3a3d01d5bb32 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:04.005819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-190aaab8-9ca9-413e-8edf-3a3d01d5bb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:04.122540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190aaab8-9ca9-413e-8edf-3a3d01d5bb32 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:04.206846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff26962-d37a-4e0a-a542-6d8a07c7ab3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfe36a14a11493397e4018b773d3cc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:04.207451 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff26962-d37a-4e0a-a542-6d8a07c7ab3c service glance] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:04.207724 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff26962-d37a-4e0a-a542-6d8a07c7ab3c service glance] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:04.208317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff26962-d37a-4e0a-a542-6d8a07c7ab3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:04.209914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eff26962-d37a-4e0a-a542-6d8a07c7ab3c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:04.268831 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:04.328695 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff26962-d37a-4e0a-a542-6d8a07c7ab3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:04.330520 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff26962-d37a-4e0a-a542-6d8a07c7ab3c service glance] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:04.350143 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 742/1475] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:06.380818 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70977145-20b9-473b-af90-2121825bffcf None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:06.381737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70977145-20b9-473b-af90-2121825bffcf None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:06.381737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70977145-20b9-473b-af90-2121825bffcf None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:06.383949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 734/1476] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:31:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:06.390630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fceb6d5-b359-4555-a8b9-bca8871115e3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:06.391316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fceb6d5-b359-4555-a8b9-bca8871115e3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:06.391493 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fceb6d5-b359-4555-a8b9-bca8871115e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:06.471915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4fceb6d5-b359-4555-a8b9-bca8871115e3 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:06.473147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4fceb6d5-b359-4555-a8b9-bca8871115e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:06.591237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 743/1477] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:06.857009 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70977145-20b9-473b-af90-2121825bffcf None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:06.858549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-70977145-20b9-473b-af90-2121825bffcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:07.000884 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70977145-20b9-473b-af90-2121825bffcf None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:07.050172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:07.104377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2d7c9-da51-4811-9ff4-7e01cc0c73a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:07.104977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2d7c9-da51-4811-9ff4-7e01cc0c73a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:07.105522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2d7c9-da51-4811-9ff4-7e01cc0c73a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:07.105790 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2d7c9-da51-4811-9ff4-7e01cc0c73a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:07.106428 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea2d7c9-da51-4811-9ff4-7e01cc0c73a2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:07.108320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: WARNING py.warnings [None req-eea2d7c9-da51-4811-9ff4-7e01cc0c73a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:31:07.108320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: warnings.warn(msg) Sep 01 16:31:07.108320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: Sep 01 16:31:07.109488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea2d7c9-da51-4811-9ff4-7e01cc0c73a2 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:07.128730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 735/1478] 10.222.0.62 () {56 vars in 1108 bytes} [Thu Sep 1 16:31:06 2022] GET /identity/v3/projects => generated 2553 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:31:09.062899 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5418f8-a328-4564-84c0-f02fab7e22f3 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:09.063815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5418f8-a328-4564-84c0-f02fab7e22f3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:09.063815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5418f8-a328-4564-84c0-f02fab7e22f3 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:09.067732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 744/1479] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:31:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:09.076172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051c168-f5f1-4e61-b8ca-e34efc60498c None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:09.076637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051c168-f5f1-4e61-b8ca-e34efc60498c None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:09.076763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051c168-f5f1-4e61-b8ca-e34efc60498c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:09.145621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e051c168-f5f1-4e61-b8ca-e34efc60498c None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:09.146884 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e051c168-f5f1-4e61-b8ca-e34efc60498c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:09.256820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 736/1480] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:09.549032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5418f8-a328-4564-84c0-f02fab7e22f3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:09.550700 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3c5418f8-a328-4564-84c0-f02fab7e22f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:09.561802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:09.712624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5418f8-a328-4564-84c0-f02fab7e22f3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:09.799739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:09.836219 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5409e4-7b30-4479-9190-234070b6c421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:09.836819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5409e4-7b30-4479-9190-234070b6c421 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:09.837067 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5409e4-7b30-4479-9190-234070b6c421 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:09.837352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5409e4-7b30-4479-9190-234070b6c421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:09.838947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dd5409e4-7b30-4479-9190-234070b6c421 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:09.934119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5409e4-7b30-4479-9190-234070b6c421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:09.935724 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5409e4-7b30-4479-9190-234070b6c421 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:09.954992 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 745/1481] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:12.095666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8da7e2-e1e2-4e21-a18e-dc4678ffb3dc None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:12.096323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8da7e2-e1e2-4e21-a18e-dc4678ffb3dc None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:12.096323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8da7e2-e1e2-4e21-a18e-dc4678ffb3dc None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:12.097629 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 737/1482] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:12.104273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc960f-d193-4ef7-a391-81ac16b4b823 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:12.104698 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc960f-d193-4ef7-a391-81ac16b4b823 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:12.104846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc960f-d193-4ef7-a391-81ac16b4b823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:12.181435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-37fc960f-d193-4ef7-a391-81ac16b4b823 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:12.182657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-37fc960f-d193-4ef7-a391-81ac16b4b823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:12.233801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:12.286075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 746/1483] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:12.586459 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8da7e2-e1e2-4e21-a18e-dc4678ffb3dc None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:12.588101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7e8da7e2-e1e2-4e21-a18e-dc4678ffb3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:12.613811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:12.720994 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8da7e2-e1e2-4e21-a18e-dc4678ffb3dc None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:12.934399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27327540-2a8f-4b66-b7e2-a15ea3a9ba74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:12.934909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27327540-2a8f-4b66-b7e2-a15ea3a9ba74 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:12.935180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27327540-2a8f-4b66-b7e2-a15ea3a9ba74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:12.935441 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27327540-2a8f-4b66-b7e2-a15ea3a9ba74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:12.937117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-27327540-2a8f-4b66-b7e2-a15ea3a9ba74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:13.035435 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:13.075909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27327540-2a8f-4b66-b7e2-a15ea3a9ba74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:13.077438 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27327540-2a8f-4b66-b7e2-a15ea3a9ba74 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:13.096578 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 738/1484] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:15.063798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ad5d-4e97-4537-872e-232a9a745a95 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:15.064758 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ad5d-4e97-4537-872e-232a9a745a95 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:15.064758 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5ad5d-4e97-4537-872e-232a9a745a95 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:15.066762 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 747/1485] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:31:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:15.073308 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653018a-9b43-4698-ba7c-650d79fc0b2d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:15.073772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653018a-9b43-4698-ba7c-650d79fc0b2d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:15.073902 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653018a-9b43-4698-ba7c-650d79fc0b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:15.145544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-5653018a-9b43-4698-ba7c-650d79fc0b2d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:15.146750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5653018a-9b43-4698-ba7c-650d79fc0b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:15.262020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 739/1486] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:15.551765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5ad5d-4e97-4537-872e-232a9a745a95 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:15.554276 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3bb5ad5d-4e97-4537-872e-232a9a745a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:15.671248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5ad5d-4e97-4537-872e-232a9a745a95 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:15.749528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:15.751787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a018df8-7109-4a8a-8c17-98a3c99a771c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:15.752439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a018df8-7109-4a8a-8c17-98a3c99a771c service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:15.752711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a018df8-7109-4a8a-8c17-98a3c99a771c service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:15.752978 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a018df8-7109-4a8a-8c17-98a3c99a771c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:15.754939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3a018df8-7109-4a8a-8c17-98a3c99a771c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:15.869361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a018df8-7109-4a8a-8c17-98a3c99a771c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:15.870859 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a018df8-7109-4a8a-8c17-98a3c99a771c service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:15.890924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 748/1487] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:17.889356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5d910-e1ed-4615-9a45-5354b600b9e2 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:17.890028 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5d910-e1ed-4615-9a45-5354b600b9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:17.890028 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5d910-e1ed-4615-9a45-5354b600b9e2 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:17.891337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 740/1488] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:17.897927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44173cc1-5bee-4413-92e3-c380e413cf08 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:17.898711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44173cc1-5bee-4413-92e3-c380e413cf08 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:17.898884 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44173cc1-5bee-4413-92e3-c380e413cf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:17.976019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-44173cc1-5bee-4413-92e3-c380e413cf08 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:17.977899 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-44173cc1-5bee-4413-92e3-c380e413cf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:18.045207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:18.093816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 749/1489] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:18.405282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5d910-e1ed-4615-9a45-5354b600b9e2 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:18.406823 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-efb5d910-e1ed-4615-9a45-5354b600b9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:18.448629 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:18.523857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5d910-e1ed-4615-9a45-5354b600b9e2 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:18.613857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c7edc-2311-41dc-b4f1-33082c4296c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:18.614471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c7edc-2311-41dc-b4f1-33082c4296c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:18.614738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c7edc-2311-41dc-b4f1-33082c4296c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:18.614995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c7edc-2311-41dc-b4f1-33082c4296c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:18.616616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-866c7edc-2311-41dc-b4f1-33082c4296c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:18.709001 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:18.739533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c7edc-2311-41dc-b4f1-33082c4296c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:18.741067 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c7edc-2311-41dc-b4f1-33082c4296c8 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:18.760903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 741/1490] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:20.950734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d2694-fffe-4359-a076-90e994f89dd0 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:20.951566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d2694-fffe-4359-a076-90e994f89dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:20.951566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d2694-fffe-4359-a076-90e994f89dd0 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:20.953860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 750/1491] 10.222.0.62 () {54 vars in 927 bytes} [Thu Sep 1 16:31:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:20.959690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97d78d-1349-409b-8bb0-b45cf2c369d1 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:20.960172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97d78d-1349-409b-8bb0-b45cf2c369d1 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:20.960282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97d78d-1349-409b-8bb0-b45cf2c369d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:21.035535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6c97d78d-1349-409b-8bb0-b45cf2c369d1 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:21.036769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6c97d78d-1349-409b-8bb0-b45cf2c369d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:21.141429 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 742/1492] 10.222.0.62 () {58 vars in 1024 bytes} [Thu Sep 1 16:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:21.426019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d2694-fffe-4359-a076-90e994f89dd0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:21.428535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9d4d2694-fffe-4359-a076-90e994f89dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:21.549180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d2694-fffe-4359-a076-90e994f89dd0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:21.628311 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27fb89-354b-423f-8e18-40f5770dc74c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:21.628927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27fb89-354b-423f-8e18-40f5770dc74c service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:21.629208 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27fb89-354b-423f-8e18-40f5770dc74c service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:21.629477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27fb89-354b-423f-8e18-40f5770dc74c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:21.631087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9f27fb89-354b-423f-8e18-40f5770dc74c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:21.640692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:21.746454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27fb89-354b-423f-8e18-40f5770dc74c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:21.747874 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27fb89-354b-423f-8e18-40f5770dc74c service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:21.767915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 751/1493] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:31.139440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=82 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:31.765351 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=50 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:35.422407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bd692-174d-4278-b287-91dd74f76c5b None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:35.423033 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bd692-174d-4278-b287-91dd74f76c5b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:35.423033 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bd692-174d-4278-b287-91dd74f76c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:35.518237 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-710bd692-174d-4278-b287-91dd74f76c5b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:35.519399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-710bd692-174d-4278-b287-91dd74f76c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:35.637179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 743/1494] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:35.645800 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44720b70-7612-498e-bad2-e38e62c29d50 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:35.646336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44720b70-7612-498e-bad2-e38e62c29d50 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:35.646336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44720b70-7612-498e-bad2-e38e62c29d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:35.730029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-44720b70-7612-498e-bad2-e38e62c29d50 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:35.731251 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-44720b70-7612-498e-bad2-e38e62c29d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:35.838927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 752/1495] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:35.844253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bd692-174d-4278-b287-91dd74f76c5b None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:35.845468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-710bd692-174d-4278-b287-91dd74f76c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:35.858682 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bd692-174d-4278-b287-91dd74f76c5b None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:35.928125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:35.963259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2382c-da22-4b5f-82ff-bed6d883c72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:35.963910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2382c-da22-4b5f-82ff-bed6d883c72f admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:35.964207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2382c-da22-4b5f-82ff-bed6d883c72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:35.964471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2382c-da22-4b5f-82ff-bed6d883c72f admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:35.965121 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2382c-da22-4b5f-82ff-bed6d883c72f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:32:35.965414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2382c-da22-4b5f-82ff-bed6d883c72f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:35.966483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: WARNING py.warnings [None req-d6d2382c-da22-4b5f-82ff-bed6d883c72f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:32:35.966483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: warnings.warn(msg) Sep 01 16:32:35.966483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: Sep 01 16:32:35.967350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2382c-da22-4b5f-82ff-bed6d883c72f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:35.978417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 744/1496] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:32:35.987621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44720b70-7612-498e-bad2-e38e62c29d50 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:35.997585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44720b70-7612-498e-bad2-e38e62c29d50 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:36.061736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:36.098530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38315c-3fb7-43c3-956c-64e2e2adeccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:36.099069 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38315c-3fb7-43c3-956c-64e2e2adeccd admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:36.099346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38315c-3fb7-43c3-956c-64e2e2adeccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:36.099608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38315c-3fb7-43c3-956c-64e2e2adeccd admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:36.100285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38315c-3fb7-43c3-956c-64e2e2adeccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:36.102026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-7c38315c-3fb7-43c3-956c-64e2e2adeccd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:32:36.102026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 01 16:32:36.102026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: Sep 01 16:32:36.103226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38315c-3fb7-43c3-956c-64e2e2adeccd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:36.161315 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 753/1497] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:35 2022] POST /identity/v3/projects => generated 379 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:36.165825 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2382c-da22-4b5f-82ff-bed6d883c72f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:36.167455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d6d2382c-da22-4b5f-82ff-bed6d883c72f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:36.203785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2382c-da22-4b5f-82ff-bed6d883c72f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:36.304681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d45ab-7103-4056-ab99-68b63bd5e273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:36.305282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d45ab-7103-4056-ab99-68b63bd5e273 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:36.305658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d45ab-7103-4056-ab99-68b63bd5e273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:36.305921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d45ab-7103-4056-ab99-68b63bd5e273 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:36.306539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d45ab-7103-4056-ab99-68b63bd5e273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:36.307567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: WARNING py.warnings [None req-a38d45ab-7103-4056-ab99-68b63bd5e273 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:32:36.307567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: warnings.warn(msg) Sep 01 16:32:36.307567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: Sep 01 16:32:36.308931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d45ab-7103-4056-ab99-68b63bd5e273 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:36.353319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 745/1498] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:36 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:36.357554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38315c-3fb7-43c3-956c-64e2e2adeccd admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:36.359313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7c38315c-3fb7-43c3-956c-64e2e2adeccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:36.422731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38315c-3fb7-43c3-956c-64e2e2adeccd admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:36.524457 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:36.536902 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7b96e-8784-4f6f-8494-7e56522923ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:36.537522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7b96e-8784-4f6f-8494-7e56522923ec admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:36.537796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7b96e-8784-4f6f-8494-7e56522923ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:36.538058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7b96e-8784-4f6f-8494-7e56522923ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:36.538714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab7b96e-8784-4f6f-8494-7e56522923ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:36.539758 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-dab7b96e-8784-4f6f-8494-7e56522923ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:32:36.539758 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 01 16:32:36.539758 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: Sep 01 16:32:36.540683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab7b96e-8784-4f6f-8494-7e56522923ec admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:36.553630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 754/1499] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:36.557935 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d45ab-7103-4056-ab99-68b63bd5e273 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:36.559633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a38d45ab-7103-4056-ab99-68b63bd5e273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:36.571587 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d45ab-7103-4056-ab99-68b63bd5e273 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:36.599242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:36.673379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfd662-bbb2-4248-8d7f-8ca0a0a492b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:36.673908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfd662-bbb2-4248-8d7f-8ca0a0a492b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:36.674180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfd662-bbb2-4248-8d7f-8ca0a0a492b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:36.674446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfd662-bbb2-4248-8d7f-8ca0a0a492b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:36.675090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dfd662-bbb2-4248-8d7f-8ca0a0a492b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:36.676140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: WARNING py.warnings [None req-b5dfd662-bbb2-4248-8d7f-8ca0a0a492b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:32:36.676140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: warnings.warn(msg) Sep 01 16:32:36.676140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: Sep 01 16:32:36.677013 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dfd662-bbb2-4248-8d7f-8ca0a0a492b7 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:36.689906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 746/1500] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:36.694570 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7b96e-8784-4f6f-8494-7e56522923ec admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:36.696124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dab7b96e-8784-4f6f-8494-7e56522923ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:36.730625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7b96e-8784-4f6f-8494-7e56522923ec admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:36.844894 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b309274-2fb4-43bb-b41d-5306134da87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:36.845573 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b309274-2fb4-43bb-b41d-5306134da87c admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:36.845841 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b309274-2fb4-43bb-b41d-5306134da87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:36.846108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b309274-2fb4-43bb-b41d-5306134da87c admin admin] PATH_INFO: `/v3/projects/b06897c733724c61bd65ad54beeae820/users/1984277d09a8470c86a71cb0bbca8c1a/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:36.869653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b309274-2fb4-43bb-b41d-5306134da87c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06897c733724c61bd65ad54beeae820, user_id=1984277d09a8470c86a71cb0bbca8c1a, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:36.870688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-3b309274-2fb4-43bb-b41d-5306134da87c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:32:36.870688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 01 16:32:36.870688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: Sep 01 16:32:36.872890 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b309274-2fb4-43bb-b41d-5306134da87c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:36.905507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 755/1501] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:36 2022] PUT /identity/v3/projects/b06897c733724c61bd65ad54beeae820/users/1984277d09a8470c86a71cb0bbca8c1a/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:36.918045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cee7de-0694-458a-8bb8-96e75c5e2bfd None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:36.918328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cee7de-0694-458a-8bb8-96e75c5e2bfd None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:36.918663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cee7de-0694-458a-8bb8-96e75c5e2bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:36.982518 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-42cee7de-0694-458a-8bb8-96e75c5e2bfd None None] MFA Rules not processed for user `1984277d09a8470c86a71cb0bbca8c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:36.983651 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-42cee7de-0694-458a-8bb8-96e75c5e2bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:37.081803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 747/1502] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:32:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:37.098847 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee014f-8924-4af6-8cd3-3dea8617de38 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:37.099134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee014f-8924-4af6-8cd3-3dea8617de38 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:37.099405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee014f-8924-4af6-8cd3-3dea8617de38 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:37.100804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 756/1503] 10.222.0.62 () {54 vars in 964 bytes} [Thu Sep 1 16:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:32:37.106026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca1d65-b02c-4dee-a3b9-6f6d122bd8ca None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:37.106340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca1d65-b02c-4dee-a3b9-6f6d122bd8ca None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:37.106580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca1d65-b02c-4dee-a3b9-6f6d122bd8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:37.137873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:37.182654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-96ca1d65-b02c-4dee-a3b9-6f6d122bd8ca None None] MFA Rules not processed for user `6160b373ace44ccfaf557b557343bf32`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:37.183855 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-96ca1d65-b02c-4dee-a3b9-6f6d122bd8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:37.286031 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 748/1504] 10.222.0.62 () {58 vars in 1061 bytes} [Thu Sep 1 16:32:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:37.295929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee014f-8924-4af6-8cd3-3dea8617de38 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:37.297530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c1ee014f-8924-4af6-8cd3-3dea8617de38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:37.351740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:37.417134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee014f-8924-4af6-8cd3-3dea8617de38 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:37.508814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c5864-efb4-435f-bd2f-d8a7393e90f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:37.509350 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c5864-efb4-435f-bd2f-d8a7393e90f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:37.509578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c5864-efb4-435f-bd2f-d8a7393e90f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:37.509877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c5864-efb4-435f-bd2f-d8a7393e90f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:37.511589 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-760c5864-efb4-435f-bd2f-d8a7393e90f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:37.630338 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c5864-efb4-435f-bd2f-d8a7393e90f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:37.631767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c5864-efb4-435f-bd2f-d8a7393e90f7 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:37.635278 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:37.652165 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 757/1505] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:37.703863 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106f8da-57cb-41dc-8dee-17b4b397b700 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:37.704426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106f8da-57cb-41dc-8dee-17b4b397b700 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:37.704426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106f8da-57cb-41dc-8dee-17b4b397b700 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:37.706024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 749/1506] 10.222.0.62 () {52 vars in 830 bytes} [Thu Sep 1 16:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:32:37.709945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c5864-efb4-435f-bd2f-d8a7393e90f7 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:37.711168 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-760c5864-efb4-435f-bd2f-d8a7393e90f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:37.815673 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c5864-efb4-435f-bd2f-d8a7393e90f7 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:37.916213 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:37.916849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:37.917119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:37.917381 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] PATH_INFO: `/v3/users/1984277d09a8470c86a71cb0bbca8c1a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:37.934586 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1984277d09a8470c86a71cb0bbca8c1a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:37.935728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:32:37.935728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 01 16:32:37.935728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: Sep 01 16:32:37.937017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:37.947672 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:38.010706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1984277d09a8470c86a71cb0bbca8c1a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:38.022709 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1984277d09a8470c86a71cb0bbca8c1a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:38.023082 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1984277d09a8470c86a71cb0bbca8c1a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:38.046819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1984277d09a8470c86a71cb0bbca8c1a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:38.061062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1984277d09a8470c86a71cb0bbca8c1a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:38.066932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 758/1507] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:32:37 2022] DELETE /identity/v3/users/1984277d09a8470c86a71cb0bbca8c1a => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:38.073591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662d3bc-f2f7-4a23-bade-142d1936e977 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:38.073953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662d3bc-f2f7-4a23-bade-142d1936e977 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:38.074312 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662d3bc-f2f7-4a23-bade-142d1936e977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:38.140383 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-5662d3bc-f2f7-4a23-bade-142d1936e977 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:38.141595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5662d3bc-f2f7-4a23-bade-142d1936e977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:38.228950 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:38.243576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 750/1508] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:38.253334 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:38.254602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:38.363470 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d7f7-7ed6-4888-b30b-7a26c914ce22 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:38.425224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:38.445928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd74478-255a-4404-9508-c45df9c7fe91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:38.446536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd74478-255a-4404-9508-c45df9c7fe91 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:38.446805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd74478-255a-4404-9508-c45df9c7fe91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:38.447067 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd74478-255a-4404-9508-c45df9c7fe91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:38.448708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3bd74478-255a-4404-9508-c45df9c7fe91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:38.563497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd74478-255a-4404-9508-c45df9c7fe91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:38.564950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd74478-255a-4404-9508-c45df9c7fe91 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:38.585324 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 759/1509] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:38.950425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662d3bc-f2f7-4a23-bade-142d1936e977 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:38.952003 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5662d3bc-f2f7-4a23-bade-142d1936e977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:38.964702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662d3bc-f2f7-4a23-bade-142d1936e977 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:39.067455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dd106-dc86-415d-8c39-87e638bcae51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:39.068099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dd106-dc86-415d-8c39-87e638bcae51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:39.068371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dd106-dc86-415d-8c39-87e638bcae51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:39.068631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dd106-dc86-415d-8c39-87e638bcae51 admin admin] PATH_INFO: `/v3/projects/b06897c733724c61bd65ad54beeae820` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:39.079156 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dd106-dc86-415d-8c39-87e638bcae51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b06897c733724c61bd65ad54beeae820)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:39.080571 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: WARNING py.warnings [None req-0a4dd106-dc86-415d-8c39-87e638bcae51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:32:39.080571 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: warnings.warn(msg) Sep 01 16:32:39.080571 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: Sep 01 16:32:39.081734 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dd106-dc86-415d-8c39-87e638bcae51 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:39.147622 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0a4dd106-dc86-415d-8c39-87e638bcae51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b06897c733724c61bd65ad54beeae820', 'request_id': 'req-0a4dd106-dc86-415d-8c39-87e638bcae51'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:39.157642 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0a4dd106-dc86-415d-8c39-87e638bcae51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b06897c733724c61bd65ad54beeae820', 'request_id': 'req-0a4dd106-dc86-415d-8c39-87e638bcae51'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:39.164363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0a4dd106-dc86-415d-8c39-87e638bcae51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b06897c733724c61bd65ad54beeae820', 'request_id': 'req-0a4dd106-dc86-415d-8c39-87e638bcae51'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:39.197924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0a4dd106-dc86-415d-8c39-87e638bcae51 admin admin] The token cache is being invalidate because project b06897c733724c61bd65ad54beeae820 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:32:39.198151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0a4dd106-dc86-415d-8c39-87e638bcae51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:39.198652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0a4dd106-dc86-415d-8c39-87e638bcae51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:32:39.200166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 751/1510] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:32:38 2022] DELETE /identity/v3/projects/b06897c733724c61bd65ad54beeae820 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:48.582415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=62 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:49.195337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:13.747767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea698e16-4506-4d54-aabf-b95e072f2789 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:13.748722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea698e16-4506-4d54-aabf-b95e072f2789 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:13.748722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea698e16-4506-4d54-aabf-b95e072f2789 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:13.750727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:34:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:13.759627 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2bd57-fa66-43e5-96cd-7c3ca8a55a46 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:13.760921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2bd57-fa66-43e5-96cd-7c3ca8a55a46 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:13.761386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2bd57-fa66-43e5-96cd-7c3ca8a55a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:13.857049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-dac2bd57-fa66-43e5-96cd-7c3ca8a55a46 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:13.858399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-dac2bd57-fa66-43e5-96cd-7c3ca8a55a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:13.973758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:14.127766 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea698e16-4506-4d54-aabf-b95e072f2789 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:14.131530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ea698e16-4506-4d54-aabf-b95e072f2789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:14.271370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea698e16-4506-4d54-aabf-b95e072f2789 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:14.278960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75711a7c-fc2e-4eec-9653-090277fc0153 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:14.279575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75711a7c-fc2e-4eec-9653-090277fc0153 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:14.279851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75711a7c-fc2e-4eec-9653-090277fc0153 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:14.280148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75711a7c-fc2e-4eec-9653-090277fc0153 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:14.289856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75711a7c-fc2e-4eec-9653-090277fc0153 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:14.291458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75711a7c-fc2e-4eec-9653-090277fc0153 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:14.301706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 761/1513] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:15.158424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27385d41-4bc8-46d8-bd06-69306b766ade None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:15.159283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27385d41-4bc8-46d8-bd06-69306b766ade None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:15.159283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27385d41-4bc8-46d8-bd06-69306b766ade None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:15.161845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:34:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:15.168306 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5a350-6952-4d56-9161-9a24adfd854d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:15.169283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5a350-6952-4d56-9161-9a24adfd854d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:15.169441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5a350-6952-4d56-9161-9a24adfd854d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:15.196963 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e2d5a350-6952-4d56-9161-9a24adfd854d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:15.198711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e2d5a350-6952-4d56-9161-9a24adfd854d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:15.208751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:15.365191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27385d41-4bc8-46d8-bd06-69306b766ade None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:15.383470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27385d41-4bc8-46d8-bd06-69306b766ade None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:15.389789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea3dcb-1105-4267-8fca-f18f9e3ac85f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:15.390384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea3dcb-1105-4267-8fca-f18f9e3ac85f service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:15.390656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea3dcb-1105-4267-8fca-f18f9e3ac85f service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:15.390922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea3dcb-1105-4267-8fca-f18f9e3ac85f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:15.399427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea3dcb-1105-4267-8fca-f18f9e3ac85f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:15.401075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea3dcb-1105-4267-8fca-f18f9e3ac85f service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:15.411066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 754/1516] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:15.630647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc4c27-38d7-4595-a1c1-ff85ea0a974f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:15.630920 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc4c27-38d7-4595-a1c1-ff85ea0a974f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:15.631184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc4c27-38d7-4595-a1c1-ff85ea0a974f None None] PATH_INFO: `/v3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:15.632695 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 763/1517] 10.222.0.74 () {54 vars in 925 bytes} [Thu Sep 1 16:34:15 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:15.639350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4767e-5788-44b8-ba74-8e4e5f461589 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:15.639700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4767e-5788-44b8-ba74-8e4e5f461589 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:15.640001 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4767e-5788-44b8-ba74-8e4e5f461589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:15.701828 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-bdb4767e-5788-44b8-ba74-8e4e5f461589 None None] MFA Rules not processed for user `6281dcf2927a4ed2870125df8801be78`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:15.702970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-bdb4767e-5788-44b8-ba74-8e4e5f461589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:15.722047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:15.787549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 755/1518] 10.222.0.74 () {58 vars in 1015 bytes} [Thu Sep 1 16:34:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:19.638009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13506c1-f0c4-4e4b-933a-56c2e895eca6 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:19.638633 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13506c1-f0c4-4e4b-933a-56c2e895eca6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:19.638633 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13506c1-f0c4-4e4b-933a-56c2e895eca6 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:19.640794 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 764/1519] 10.222.0.62 () {54 vars in 961 bytes} [Thu Sep 1 16:34:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:19.648768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad2f40-50e9-4c2c-a168-b7814dbf331e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:19.649223 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad2f40-50e9-4c2c-a168-b7814dbf331e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:19.649330 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad2f40-50e9-4c2c-a168-b7814dbf331e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:19.676913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-b1ad2f40-50e9-4c2c-a168-b7814dbf331e None None] MFA Rules not processed for user `6281dcf2927a4ed2870125df8801be78`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:19.678165 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b1ad2f40-50e9-4c2c-a168-b7814dbf331e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:19.688687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 756/1520] 10.222.0.62 () {58 vars in 1058 bytes} [Thu Sep 1 16:34:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:19.696532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13506c1-f0c4-4e4b-933a-56c2e895eca6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:19.712267 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13506c1-f0c4-4e4b-933a-56c2e895eca6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:19.717972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd46cee-59c1-4f64-9e69-013d24e4f287 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6281dcf2927a4ed2870125df8801be78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:19.718578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd46cee-59c1-4f64-9e69-013d24e4f287 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:19.718846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd46cee-59c1-4f64-9e69-013d24e4f287 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:19.719134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd46cee-59c1-4f64-9e69-013d24e4f287 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:19.727459 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd46cee-59c1-4f64-9e69-013d24e4f287 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:19.729062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd46cee-59c1-4f64-9e69-013d24e4f287 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:19.738697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 765/1521] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:26.658641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093d217-254d-4669-8d13-a0276e375a36 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:26.659287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093d217-254d-4669-8d13-a0276e375a36 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:26.659287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093d217-254d-4669-8d13-a0276e375a36 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:26.661365 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:26.666165 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b7fb4-0c24-4d08-a073-7b8f6d72368b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:26.666656 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b7fb4-0c24-4d08-a073-7b8f6d72368b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:26.666656 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b7fb4-0c24-4d08-a073-7b8f6d72368b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:26.694184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-167b7fb4-0c24-4d08-a073-7b8f6d72368b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:26.695407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-167b7fb4-0c24-4d08-a073-7b8f6d72368b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:26.706101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:26.867575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093d217-254d-4669-8d13-a0276e375a36 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:26.883505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093d217-254d-4669-8d13-a0276e375a36 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:26.890174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d3c41-ca82-474e-b006-7cdc9e342888 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:26.890796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d3c41-ca82-474e-b006-7cdc9e342888 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:26.891080 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d3c41-ca82-474e-b006-7cdc9e342888 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:26.891356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d3c41-ca82-474e-b006-7cdc9e342888 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:26.899853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d3c41-ca82-474e-b006-7cdc9e342888 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:26.901504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d3c41-ca82-474e-b006-7cdc9e342888 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:26.911753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 758/1524] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:27.753743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1139c-f849-430c-b870-20b4c02995d0 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:27.754291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1139c-f849-430c-b870-20b4c02995d0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:27.754291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1139c-f849-430c-b870-20b4c02995d0 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:27.755621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:27.763032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca07c0f-f2cf-48db-9fb5-dfc6abcc801b None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:27.763514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca07c0f-f2cf-48db-9fb5-dfc6abcc801b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:27.763514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca07c0f-f2cf-48db-9fb5-dfc6abcc801b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:27.786586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-aca07c0f-f2cf-48db-9fb5-dfc6abcc801b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:27.787804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-aca07c0f-f2cf-48db-9fb5-dfc6abcc801b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:27.797236 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:27.959975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1139c-f849-430c-b870-20b4c02995d0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:27.978689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1139c-f849-430c-b870-20b4c02995d0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:27.985722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f2bf7-1920-4148-898e-1f9148ca1088 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:27.986322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f2bf7-1920-4148-898e-1f9148ca1088 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:27.986595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f2bf7-1920-4148-898e-1f9148ca1088 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:27.986862 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f2bf7-1920-4148-898e-1f9148ca1088 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:27.995865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f2bf7-1920-4148-898e-1f9148ca1088 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:27.997545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f2bf7-1920-4148-898e-1f9148ca1088 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:28.007851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 768/1527] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:31.797028 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a1859-e17c-46b0-93b7-85a1b9d40edd None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.797645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a1859-e17c-46b0-93b7-85a1b9d40edd None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.797645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a1859-e17c-46b0-93b7-85a1b9d40edd None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.798965 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:31.805273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972f9b2-9871-42c0-a3a4-43391af3af86 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.805766 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972f9b2-9871-42c0-a3a4-43391af3af86 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.805867 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972f9b2-9871-42c0-a3a4-43391af3af86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.827762 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1972f9b2-9871-42c0-a3a4-43391af3af86 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:31.829000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1972f9b2-9871-42c0-a3a4-43391af3af86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:31.839226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:32.128888 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a1859-e17c-46b0-93b7-85a1b9d40edd None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:32.131063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-889a1859-e17c-46b0-93b7-85a1b9d40edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:32.216112 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a1859-e17c-46b0-93b7-85a1b9d40edd None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:32.220613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4e1a9-e4e0-4aa9-a2e6-b5fa047f1539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:32.221230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4e1a9-e4e0-4aa9-a2e6-b5fa047f1539 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:32.221506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4e1a9-e4e0-4aa9-a2e6-b5fa047f1539 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:32.221770 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4e1a9-e4e0-4aa9-a2e6-b5fa047f1539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:32.233345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4e1a9-e4e0-4aa9-a2e6-b5fa047f1539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:32.234859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4e1a9-e4e0-4aa9-a2e6-b5fa047f1539 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:32.244740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 761/1530] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:34.139923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ebdcc-da14-4b5e-8da3-fd0158c2b39b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:34.140527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ebdcc-da14-4b5e-8da3-fd0158c2b39b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:34.140527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ebdcc-da14-4b5e-8da3-fd0158c2b39b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:34.141795 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:34.147482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29974be0-e4e8-4f17-a374-c24386b63a67 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:34.147990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29974be0-e4e8-4f17-a374-c24386b63a67 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:34.148144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29974be0-e4e8-4f17-a374-c24386b63a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:34.172931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-29974be0-e4e8-4f17-a374-c24386b63a67 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:34.174702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-29974be0-e4e8-4f17-a374-c24386b63a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:34.188171 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:34.475754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ebdcc-da14-4b5e-8da3-fd0158c2b39b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:34.491356 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ebdcc-da14-4b5e-8da3-fd0158c2b39b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:34.495841 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e15623-6402-4116-b282-3047358bece5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:34.496470 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e15623-6402-4116-b282-3047358bece5 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:34.496745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e15623-6402-4116-b282-3047358bece5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:34.497007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e15623-6402-4116-b282-3047358bece5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:34.508076 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e15623-6402-4116-b282-3047358bece5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:34.509503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e15623-6402-4116-b282-3047358bece5 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:34.519974 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 771/1533] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:38.096749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197c888-66e9-4630-b3fb-3a214716e5b5 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.097633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197c888-66e9-4630-b3fb-3a214716e5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.097633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197c888-66e9-4630-b3fb-3a214716e5b5 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.099735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:34:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:38.105248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4de4d8-580f-40ad-b112-4dab9eb2a4c3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.105689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4de4d8-580f-40ad-b112-4dab9eb2a4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.105803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4de4d8-580f-40ad-b112-4dab9eb2a4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.127785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9a4de4d8-580f-40ad-b112-4dab9eb2a4c3 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:38.129009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9a4de4d8-580f-40ad-b112-4dab9eb2a4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:38.138697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:38.716757 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197c888-66e9-4630-b3fb-3a214716e5b5 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:38.718825 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b197c888-66e9-4630-b3fb-3a214716e5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:38.796738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:38.812705 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197c888-66e9-4630-b3fb-3a214716e5b5 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:38.816770 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e11d7-647a-4e79-a66c-1fae14e2ed06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:38.817379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e11d7-647a-4e79-a66c-1fae14e2ed06 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.817650 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e11d7-647a-4e79-a66c-1fae14e2ed06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.817913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e11d7-647a-4e79-a66c-1fae14e2ed06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.836335 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e11d7-647a-4e79-a66c-1fae14e2ed06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:38.838701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e11d7-647a-4e79-a66c-1fae14e2ed06 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:38.856339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 764/1536] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:40.796677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde6adb-1926-41e6-a9e9-61de082522f8 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:40.797236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde6adb-1926-41e6-a9e9-61de082522f8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:40.797236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde6adb-1926-41e6-a9e9-61de082522f8 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:40.798579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:40.804053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d388-d2f8-48d2-bac8-63c3d55600cd None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:40.804469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d388-d2f8-48d2-bac8-63c3d55600cd None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:40.804594 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d388-d2f8-48d2-bac8-63c3d55600cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:40.826214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-1383d388-d2f8-48d2-bac8-63c3d55600cd None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:40.827443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1383d388-d2f8-48d2-bac8-63c3d55600cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:40.837654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:41.451091 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde6adb-1926-41e6-a9e9-61de082522f8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:41.453134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6dde6adb-1926-41e6-a9e9-61de082522f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:41.461932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=84 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:41.472698 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde6adb-1926-41e6-a9e9-61de082522f8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:41.477940 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54673694-f9fc-40f7-a2bc-20f3082d9bea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:41.478537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54673694-f9fc-40f7-a2bc-20f3082d9bea service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:41.478806 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54673694-f9fc-40f7-a2bc-20f3082d9bea service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:41.479070 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54673694-f9fc-40f7-a2bc-20f3082d9bea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:41.489174 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54673694-f9fc-40f7-a2bc-20f3082d9bea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:41.490586 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54673694-f9fc-40f7-a2bc-20f3082d9bea service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:41.500855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 774/1539] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:50.820563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=11 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:51.496092 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:05.817196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6575b0-eb94-4511-afbb-0bc5a6cb187d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:05.817845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6575b0-eb94-4511-afbb-0bc5a6cb187d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:05.817845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6575b0-eb94-4511-afbb-0bc5a6cb187d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:05.839289 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a666b2-4845-416b-be41-f9f808c2fc06 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:05.839911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a666b2-4845-416b-be41-f9f808c2fc06 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:05.839911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a666b2-4845-416b-be41-f9f808c2fc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:05.847064 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-ad6575b0-eb94-4511-afbb-0bc5a6cb187d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:05.848362 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ad6575b0-eb94-4511-afbb-0bc5a6cb187d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:05.863199 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 766/1540] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:05.879090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-76a666b2-4845-416b-be41-f9f808c2fc06 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:05.879702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecc3ea-3289-4a8c-a7f1-6ba9e3fc121d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:05.880020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecc3ea-3289-4a8c-a7f1-6ba9e3fc121d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:05.880299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecc3ea-3289-4a8c-a7f1-6ba9e3fc121d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:05.880431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-76a666b2-4845-416b-be41-f9f808c2fc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:05.892579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 775/1541] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:05.901905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49360-a364-46dd-861c-d9c5e71b6b61 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:05.902294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49360-a364-46dd-861c-d9c5e71b6b61 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:05.902569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49360-a364-46dd-861c-d9c5e71b6b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:05.921195 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-30ecc3ea-3289-4a8c-a7f1-6ba9e3fc121d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:05.922609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-30ecc3ea-3289-4a8c-a7f1-6ba9e3fc121d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:05.937957 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-88f49360-a364-46dd-861c-d9c5e71b6b61 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:05.938705 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 767/1542] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 01 16:36:05.939183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-88f49360-a364-46dd-861c-d9c5e71b6b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:05.947009 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecc3ea-3289-4a8c-a7f1-6ba9e3fc121d None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:05.960563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 776/1543] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:05.964377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecc3ea-3289-4a8c-a7f1-6ba9e3fc121d None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:05.965603 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f49360-a364-46dd-861c-d9c5e71b6b61 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:05.982879 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f49360-a364-46dd-861c-d9c5e71b6b61 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:05.984718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcaf20-0176-4d29-b336-7389563bc311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:05.985370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcaf20-0176-4d29-b336-7389563bc311 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:05.985651 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcaf20-0176-4d29-b336-7389563bc311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:05.985923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcaf20-0176-4d29-b336-7389563bc311 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:05.986585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcaf20-0176-4d29-b336-7389563bc311 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:05.986885 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcaf20-0176-4d29-b336-7389563bc311 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:05.988971 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcaf20-0176-4d29-b336-7389563bc311 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:05.997857 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d2b11-2090-4e15-b86c-108124580061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:05.998517 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d2b11-2090-4e15-b86c-108124580061 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:05.998823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d2b11-2090-4e15-b86c-108124580061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:05.999057 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d2b11-2090-4e15-b86c-108124580061 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:05.999725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d2b11-2090-4e15-b86c-108124580061 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:06.000058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d2b11-2090-4e15-b86c-108124580061 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.001668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-111d2b11-2090-4e15-b86c-108124580061 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:36:06.001668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 01 16:36:06.001668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: Sep 01 16:36:06.002562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d2b11-2090-4e15-b86c-108124580061 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.020701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 768/1544] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:36:06.032313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 777/1545] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:36:06.034827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcaf20-0176-4d29-b336-7389563bc311 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.041352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d2b11-2090-4e15-b86c-108124580061 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.059698 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d2b11-2090-4e15-b86c-108124580061 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.061710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcaf20-0176-4d29-b336-7389563bc311 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.069835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b90ab2-c897-4b3d-834f-796e8ec500f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.071933 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b90ab2-c897-4b3d-834f-796e8ec500f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.072239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b90ab2-c897-4b3d-834f-796e8ec500f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.072522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b90ab2-c897-4b3d-834f-796e8ec500f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.073259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b90ab2-c897-4b3d-834f-796e8ec500f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.075281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b90ab2-c897-4b3d-834f-796e8ec500f4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.088739 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1440fa-5612-4cbb-ad84-c33f58f97a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.089602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1440fa-5612-4cbb-ad84-c33f58f97a65 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.089882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1440fa-5612-4cbb-ad84-c33f58f97a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.090147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1440fa-5612-4cbb-ad84-c33f58f97a65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.090802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1440fa-5612-4cbb-ad84-c33f58f97a65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.092212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: WARNING py.warnings [None req-6b1440fa-5612-4cbb-ad84-c33f58f97a65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:36:06.092212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: warnings.warn(msg) Sep 01 16:36:06.092212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: Sep 01 16:36:06.093493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1440fa-5612-4cbb-ad84-c33f58f97a65 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.157668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 778/1546] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:36:06.159591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55345dd-80fa-411b-b45a-4bc81af0ff53 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.160132 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55345dd-80fa-411b-b45a-4bc81af0ff53 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.160624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55345dd-80fa-411b-b45a-4bc81af0ff53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.182421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 769/1547] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:36:06.184242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91824e5c-1a6c-4a78-bcc0-17caeba1b5ce None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.185287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91824e5c-1a6c-4a78-bcc0-17caeba1b5ce None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.185778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91824e5c-1a6c-4a78-bcc0-17caeba1b5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.195840 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f55345dd-80fa-411b-b45a-4bc81af0ff53 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:06.197528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f55345dd-80fa-411b-b45a-4bc81af0ff53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.229639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-91824e5c-1a6c-4a78-bcc0-17caeba1b5ce None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:06.231381 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-91824e5c-1a6c-4a78-bcc0-17caeba1b5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.288043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 779/1548] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.288829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55345dd-80fa-411b-b45a-4bc81af0ff53 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.298566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55345dd-80fa-411b-b45a-4bc81af0ff53 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.303422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd47ef-76f6-4335-bec8-08ff44f9816f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.304007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd47ef-76f6-4335-bec8-08ff44f9816f admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.304285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd47ef-76f6-4335-bec8-08ff44f9816f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.304552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd47ef-76f6-4335-bec8-08ff44f9816f admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.305211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd47ef-76f6-4335-bec8-08ff44f9816f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.306467 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-fbfd47ef-76f6-4335-bec8-08ff44f9816f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:36:06.306467 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 01 16:36:06.306467 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: Sep 01 16:36:06.307679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd47ef-76f6-4335-bec8-08ff44f9816f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.315991 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 770/1549] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.318161 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e186b8-c1fc-4ff0-9b76-e28773356190 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.318943 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e186b8-c1fc-4ff0-9b76-e28773356190 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.319730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e186b8-c1fc-4ff0-9b76-e28773356190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.335804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 780/1550] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/users => generated 292 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:06.336900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd47ef-76f6-4335-bec8-08ff44f9816f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.342939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e8e186b8-c1fc-4ff0-9b76-e28773356190 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:06.344537 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e8e186b8-c1fc-4ff0-9b76-e28773356190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.348388 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd47ef-76f6-4335-bec8-08ff44f9816f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.353501 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 771/1551] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.353664 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac2a5f-823d-408b-a934-88f8f8bb5c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.354190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac2a5f-823d-408b-a934-88f8f8bb5c74 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.354470 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac2a5f-823d-408b-a934-88f8f8bb5c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.354741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac2a5f-823d-408b-a934-88f8f8bb5c74 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.355371 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac2a5f-823d-408b-a934-88f8f8bb5c74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.356063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b79b5-8d1c-4689-bfb2-5d34e45e2af4 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.356547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b79b5-8d1c-4689-bfb2-5d34e45e2af4 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.357220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b79b5-8d1c-4689-bfb2-5d34e45e2af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.357407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac2a5f-823d-408b-a934-88f8f8bb5c74 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.379797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6c0b79b5-8d1c-4689-bfb2-5d34e45e2af4 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:06.381040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6c0b79b5-8d1c-4689-bfb2-5d34e45e2af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.381224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 781/1552] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/users => generated 294 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:06.382604 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df39276-4239-44f8-9f71-aa8d4ad57cf5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.382889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df39276-4239-44f8-9f71-aa8d4ad57cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.383160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df39276-4239-44f8-9f71-aa8d4ad57cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.392288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 772/1553] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.393598 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed825af-09b7-4147-9345-e6a2cd85ee95 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.393877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed825af-09b7-4147-9345-e6a2cd85ee95 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.394289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed825af-09b7-4147-9345-e6a2cd85ee95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.405421 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9df39276-4239-44f8-9f71-aa8d4ad57cf5 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:06.406752 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9df39276-4239-44f8-9f71-aa8d4ad57cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.414430 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-5ed825af-09b7-4147-9345-e6a2cd85ee95 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:06.415412 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 782/1554] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.415565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5ed825af-09b7-4147-9345-e6a2cd85ee95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.416613 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46724b34-83ea-46ad-b047-960161c95e0c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.416906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46724b34-83ea-46ad-b047-960161c95e0c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.417158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46724b34-83ea-46ad-b047-960161c95e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.423874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 773/1555] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.424715 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed825af-09b7-4147-9345-e6a2cd85ee95 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.433969 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed825af-09b7-4147-9345-e6a2cd85ee95 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.437305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-46724b34-83ea-46ad-b047-960161c95e0c None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:06.438602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-46724b34-83ea-46ad-b047-960161c95e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.438819 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdb98f-bfb8-4432-aa1d-ae7b83ce62da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.439415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb98f-bfb8-4432-aa1d-ae7b83ce62da admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.439684 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb98f-bfb8-4432-aa1d-ae7b83ce62da admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.439957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb98f-bfb8-4432-aa1d-ae7b83ce62da admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.440609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdb98f-bfb8-4432-aa1d-ae7b83ce62da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.442177 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdb98f-bfb8-4432-aa1d-ae7b83ce62da admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.447528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 783/1556] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.448833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bbc8c-1659-49ae-9dc3-4993c83e78b3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.449119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bbc8c-1659-49ae-9dc3-4993c83e78b3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.449388 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bbc8c-1659-49ae-9dc3-4993c83e78b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.453084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 774/1557] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:06.454610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3495526-0b3d-4664-a390-3d37ecf57045 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.454900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3495526-0b3d-4664-a390-3d37ecf57045 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.455166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3495526-0b3d-4664-a390-3d37ecf57045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.472732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-846bbc8c-1659-49ae-9dc3-4993c83e78b3 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:06.473771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-f3495526-0b3d-4664-a390-3d37ecf57045 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:06.473892 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-846bbc8c-1659-49ae-9dc3-4993c83e78b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.474921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f3495526-0b3d-4664-a390-3d37ecf57045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.482042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 784/1558] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.482904 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bbc8c-1659-49ae-9dc3-4993c83e78b3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.483492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 775/1559] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.484779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19cd60-fcca-4009-b27f-665ae68d4604 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.485065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19cd60-fcca-4009-b27f-665ae68d4604 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.485337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19cd60-fcca-4009-b27f-665ae68d4604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.496449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bbc8c-1659-49ae-9dc3-4993c83e78b3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.501377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33678e-227f-4d5a-8ea8-092f2a82ea4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.502010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33678e-227f-4d5a-8ea8-092f2a82ea4d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.502382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33678e-227f-4d5a-8ea8-092f2a82ea4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.502728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33678e-227f-4d5a-8ea8-092f2a82ea4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.503385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33678e-227f-4d5a-8ea8-092f2a82ea4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.504753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-8c19cd60-fcca-4009-b27f-665ae68d4604 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:06.504988 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33678e-227f-4d5a-8ea8-092f2a82ea4d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.506054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8c19cd60-fcca-4009-b27f-665ae68d4604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.514283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 776/1560] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.515647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 785/1561] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:06.516530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33678e-227f-4d5a-8ea8-092f2a82ea4d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.519103 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100616a-2746-418d-9145-bc5fa1974231 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.519543 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100616a-2746-418d-9145-bc5fa1974231 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.519678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100616a-2746-418d-9145-bc5fa1974231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.526675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33678e-227f-4d5a-8ea8-092f2a82ea4d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.531657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca8b8f-cd9f-4b77-8d18-760dc62f3ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.532368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca8b8f-cd9f-4b77-8d18-760dc62f3ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.532643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca8b8f-cd9f-4b77-8d18-760dc62f3ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.532908 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca8b8f-cd9f-4b77-8d18-760dc62f3ec7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.534261 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca8b8f-cd9f-4b77-8d18-760dc62f3ec7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:06.534564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca8b8f-cd9f-4b77-8d18-760dc62f3ec7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.536175 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca8b8f-cd9f-4b77-8d18-760dc62f3ec7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.539424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-1100616a-2746-418d-9145-bc5fa1974231 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:06.540594 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1100616a-2746-418d-9145-bc5fa1974231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.549274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 777/1562] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.556714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 786/1563] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:36:06.556873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100616a-2746-418d-9145-bc5fa1974231 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.556960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca8b8f-cd9f-4b77-8d18-760dc62f3ec7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.562848 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100616a-2746-418d-9145-bc5fa1974231 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.565976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca8b8f-cd9f-4b77-8d18-760dc62f3ec7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.567575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511874c-36c3-49e5-8bd0-d45a011c6f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.568222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511874c-36c3-49e5-8bd0-d45a011c6f6d admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.568498 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511874c-36c3-49e5-8bd0-d45a011c6f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.568767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511874c-36c3-49e5-8bd0-d45a011c6f6d admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.569418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511874c-36c3-49e5-8bd0-d45a011c6f6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:06.569718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511874c-36c3-49e5-8bd0-d45a011c6f6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.571110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f910320-b47a-4a22-9dfd-34760346ac52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.571324 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511874c-36c3-49e5-8bd0-d45a011c6f6d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.571702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f910320-b47a-4a22-9dfd-34760346ac52 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.572019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f910320-b47a-4a22-9dfd-34760346ac52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.572283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f910320-b47a-4a22-9dfd-34760346ac52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.572935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f910320-b47a-4a22-9dfd-34760346ac52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.574496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f910320-b47a-4a22-9dfd-34760346ac52 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.583812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 778/1564] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:36:06.584736 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511874c-36c3-49e5-8bd0-d45a011c6f6d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.588728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 787/1565] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:06.588728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bdbe1-b50b-4d85-8f1f-56c9f8d6991d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.588728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bdbe1-b50b-4d85-8f1f-56c9f8d6991d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.588728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bdbe1-b50b-4d85-8f1f-56c9f8d6991d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.600146 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511874c-36c3-49e5-8bd0-d45a011c6f6d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.604873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3667bb-ffa7-49a5-865b-d9ba84f53894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.605481 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3667bb-ffa7-49a5-865b-d9ba84f53894 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.605754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3667bb-ffa7-49a5-865b-d9ba84f53894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.606021 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3667bb-ffa7-49a5-865b-d9ba84f53894 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.606654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3667bb-ffa7-49a5-865b-d9ba84f53894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:06.607183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bf4bdbe1-b50b-4d85-8f1f-56c9f8d6991d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:06.607785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3667bb-ffa7-49a5-865b-d9ba84f53894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.608510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bf4bdbe1-b50b-4d85-8f1f-56c9f8d6991d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:06.609465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3667bb-ffa7-49a5-865b-d9ba84f53894 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.616346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 788/1566] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:06.620822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bdbe1-b50b-4d85-8f1f-56c9f8d6991d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.621652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 779/1567] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:36:06.622670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3667bb-ffa7-49a5-865b-d9ba84f53894 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.630789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bdbe1-b50b-4d85-8f1f-56c9f8d6991d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.633645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3667bb-ffa7-49a5-865b-d9ba84f53894 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.635311 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe7cfb-d286-4918-9f48-ef623c33eedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.635911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe7cfb-d286-4918-9f48-ef623c33eedf admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.636210 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe7cfb-d286-4918-9f48-ef623c33eedf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.636475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe7cfb-d286-4918-9f48-ef623c33eedf admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.637122 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe7cfb-d286-4918-9f48-ef623c33eedf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:06.637420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe7cfb-d286-4918-9f48-ef623c33eedf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.639150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe7cfb-d286-4918-9f48-ef623c33eedf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.639465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140867d-8e26-4453-b0be-2ef803985867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.640106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140867d-8e26-4453-b0be-2ef803985867 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.640386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140867d-8e26-4453-b0be-2ef803985867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.640670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140867d-8e26-4453-b0be-2ef803985867 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.641279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140867d-8e26-4453-b0be-2ef803985867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.642158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:06.643021 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140867d-8e26-4453-b0be-2ef803985867 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.652560 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 789/1568] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:36:06.653496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe7cfb-d286-4918-9f48-ef623c33eedf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.656853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 780/1569] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:06.657738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140867d-8e26-4453-b0be-2ef803985867 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.664011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe7cfb-d286-4918-9f48-ef623c33eedf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.667673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140867d-8e26-4453-b0be-2ef803985867 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.668952 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3761b-6ac3-4ad3-9dfd-3534af35de19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.669686 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3761b-6ac3-4ad3-9dfd-3534af35de19 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.670039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3761b-6ac3-4ad3-9dfd-3534af35de19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.670221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3761b-6ac3-4ad3-9dfd-3534af35de19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.670896 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3761b-6ac3-4ad3-9dfd-3534af35de19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:06.671195 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3761b-6ac3-4ad3-9dfd-3534af35de19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.672823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3761b-6ac3-4ad3-9dfd-3534af35de19 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.673312 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb2a9c-7a74-498c-8ecb-78e6d9cc1f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.673905 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb2a9c-7a74-498c-8ecb-78e6d9cc1f5a admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.674182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb2a9c-7a74-498c-8ecb-78e6d9cc1f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.674451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb2a9c-7a74-498c-8ecb-78e6d9cc1f5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.675078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb2a9c-7a74-498c-8ecb-78e6d9cc1f5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.677036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb2a9c-7a74-498c-8ecb-78e6d9cc1f5a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.684782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 790/1570] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:36:06.685603 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3761b-6ac3-4ad3-9dfd-3534af35de19 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.696218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3761b-6ac3-4ad3-9dfd-3534af35de19 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.701344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e363a-8198-412c-a72a-52a3586dff9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.702120 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e363a-8198-412c-a72a-52a3586dff9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.702433 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e363a-8198-412c-a72a-52a3586dff9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.702745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e363a-8198-412c-a72a-52a3586dff9e admin admin] PATH_INFO: `/v3/projects/276464f90d71465097dae2fe81bdf031/users/f1c0a7b17511415ea9316dc8e61dc8c9/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.716021 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 781/1571] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:06.716021 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb2a9c-7a74-498c-8ecb-78e6d9cc1f5a admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.718390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:06.719117 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e363a-8198-412c-a72a-52a3586dff9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=276464f90d71465097dae2fe81bdf031, user_id=f1c0a7b17511415ea9316dc8e61dc8c9, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.722108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e363a-8198-412c-a72a-52a3586dff9e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.735962 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 791/1572] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:06 2022] PUT /identity/v3/projects/276464f90d71465097dae2fe81bdf031/users/f1c0a7b17511415ea9316dc8e61dc8c9/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:06.736798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e363a-8198-412c-a72a-52a3586dff9e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.779946 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb2a9c-7a74-498c-8ecb-78e6d9cc1f5a admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.808311 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e363a-8198-412c-a72a-52a3586dff9e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.813031 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88f7b1-854b-482f-a032-9a3fbcf44f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.813642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88f7b1-854b-482f-a032-9a3fbcf44f26 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.813915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88f7b1-854b-482f-a032-9a3fbcf44f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.814181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88f7b1-854b-482f-a032-9a3fbcf44f26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.814823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88f7b1-854b-482f-a032-9a3fbcf44f26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:06.815120 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88f7b1-854b-482f-a032-9a3fbcf44f26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.816754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88f7b1-854b-482f-a032-9a3fbcf44f26 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.829046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 792/1573] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:36:06.829046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88f7b1-854b-482f-a032-9a3fbcf44f26 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.839670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c1550-1b41-4154-a012-67bab94770c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.840403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c1550-1b41-4154-a012-67bab94770c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.840682 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c1550-1b41-4154-a012-67bab94770c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.840951 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c1550-1b41-4154-a012-67bab94770c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.842044 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c1550-1b41-4154-a012-67bab94770c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.844100 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c1550-1b41-4154-a012-67bab94770c7 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.844574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88f7b1-854b-482f-a032-9a3fbcf44f26 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.849380 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731496b3-6ce8-4cb7-a9cb-e4cdde5121d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.849984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731496b3-6ce8-4cb7-a9cb-e4cdde5121d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.850258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731496b3-6ce8-4cb7-a9cb-e4cdde5121d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.850525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731496b3-6ce8-4cb7-a9cb-e4cdde5121d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.851150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731496b3-6ce8-4cb7-a9cb-e4cdde5121d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.853128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731496b3-6ce8-4cb7-a9cb-e4cdde5121d0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.871768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 782/1574] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/projects => generated 391 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:06.873087 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c1550-1b41-4154-a012-67bab94770c7 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.888587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 793/1575] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:06.889489 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731496b3-6ce8-4cb7-a9cb-e4cdde5121d0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.939110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c1550-1b41-4154-a012-67bab94770c7 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.958718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731496b3-6ce8-4cb7-a9cb-e4cdde5121d0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:06.963865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121fcbc-7d57-4311-adaf-507889cb5904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:06.964502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121fcbc-7d57-4311-adaf-507889cb5904 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:06.964777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121fcbc-7d57-4311-adaf-507889cb5904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:06.965040 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121fcbc-7d57-4311-adaf-507889cb5904 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:06.965723 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121fcbc-7d57-4311-adaf-507889cb5904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:06.967731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121fcbc-7d57-4311-adaf-507889cb5904 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:06.980670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:06.996448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 794/1576] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/projects => generated 357 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:06.997439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121fcbc-7d57-4311-adaf-507889cb5904 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:06.999320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df816b-1206-41a6-ba3a-5c201556455f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.000131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df816b-1206-41a6-ba3a-5c201556455f admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.000393 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df816b-1206-41a6-ba3a-5c201556455f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.000711 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df816b-1206-41a6-ba3a-5c201556455f admin admin] PATH_INFO: `/v3/projects/6470f32e4a954dcab3e0ccdd2736c1f6/users/849f875ede78424c81919df6d9c152dd/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.010248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:07.015775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df816b-1206-41a6-ba3a-5c201556455f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6470f32e4a954dcab3e0ccdd2736c1f6, user_id=849f875ede78424c81919df6d9c152dd, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.016881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: WARNING py.warnings [None req-b5df816b-1206-41a6-ba3a-5c201556455f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:36:07.016881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: warnings.warn(msg) Sep 01 16:36:07.016881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: Sep 01 16:36:07.018867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df816b-1206-41a6-ba3a-5c201556455f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.033034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 783/1577] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:06 2022] PUT /identity/v3/projects/6470f32e4a954dcab3e0ccdd2736c1f6/users/849f875ede78424c81919df6d9c152dd/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:07.033831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df816b-1206-41a6-ba3a-5c201556455f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.062336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121fcbc-7d57-4311-adaf-507889cb5904 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.100836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df816b-1206-41a6-ba3a-5c201556455f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.105681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fffbc-b84c-4b89-938e-67ad4144215d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.106339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fffbc-b84c-4b89-938e-67ad4144215d admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.106608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fffbc-b84c-4b89-938e-67ad4144215d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.107139 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fffbc-b84c-4b89-938e-67ad4144215d admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.107785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9fffbc-b84c-4b89-938e-67ad4144215d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.109750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9fffbc-b84c-4b89-938e-67ad4144215d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.119397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c2a20-aa98-4be1-9a9f-207079dc7b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.120056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c2a20-aa98-4be1-9a9f-207079dc7b06 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.120332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c2a20-aa98-4be1-9a9f-207079dc7b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.120840 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c2a20-aa98-4be1-9a9f-207079dc7b06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.121217 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c2a20-aa98-4be1-9a9f-207079dc7b06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.123881 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c2a20-aa98-4be1-9a9f-207079dc7b06 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.131959 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 784/1578] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:07 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:07.133828 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e4446-2dcd-4b3b-91e9-c9a03c0d5708 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.134271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e4446-2dcd-4b3b-91e9-c9a03c0d5708 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.134714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e4446-2dcd-4b3b-91e9-c9a03c0d5708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.154693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 795/1579] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:06 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:07.156905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c2a20-aa98-4be1-9a9f-207079dc7b06 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.165860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-938e4446-2dcd-4b3b-91e9-c9a03c0d5708 None None] MFA Rules not processed for user `f1c0a7b17511415ea9316dc8e61dc8c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:07.167081 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-938e4446-2dcd-4b3b-91e9-c9a03c0d5708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:07.218852 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c2a20-aa98-4be1-9a9f-207079dc7b06 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.223687 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed0fec-1a71-492e-83db-2e7133d362a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.224451 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed0fec-1a71-492e-83db-2e7133d362a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.224693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed0fec-1a71-492e-83db-2e7133d362a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.224950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed0fec-1a71-492e-83db-2e7133d362a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.225582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed0fec-1a71-492e-83db-2e7133d362a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.227519 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed0fec-1a71-492e-83db-2e7133d362a2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.256490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 785/1580] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:07.257637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e4446-2dcd-4b3b-91e9-c9a03c0d5708 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.263539 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 796/1581] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:07 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:07.264542 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed0fec-1a71-492e-83db-2e7133d362a2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.273457 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e4446-2dcd-4b3b-91e9-c9a03c0d5708 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.307479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:07.320729 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:07.337384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89712a4-16ad-45a1-bf9b-8f920d9e8c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.338013 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89712a4-16ad-45a1-bf9b-8f920d9e8c7d admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.338290 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89712a4-16ad-45a1-bf9b-8f920d9e8c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.338672 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89712a4-16ad-45a1-bf9b-8f920d9e8c7d admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.339328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89712a4-16ad-45a1-bf9b-8f920d9e8c7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.340035 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed0fec-1a71-492e-83db-2e7133d362a2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.341359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89712a4-16ad-45a1-bf9b-8f920d9e8c7d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.345380 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eefe9e-5177-4e01-af6e-cb9c32792907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.346042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eefe9e-5177-4e01-af6e-cb9c32792907 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.346320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eefe9e-5177-4e01-af6e-cb9c32792907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.346589 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eefe9e-5177-4e01-af6e-cb9c32792907 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.347213 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eefe9e-5177-4e01-af6e-cb9c32792907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.349177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eefe9e-5177-4e01-af6e-cb9c32792907 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.363666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 786/1582] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:07 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:07.368059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89712a4-16ad-45a1-bf9b-8f920d9e8c7d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.373317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 797/1583] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:07 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:07.374765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e4c53-6b09-4a41-80d3-ecdd719d34b3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.375044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e4c53-6b09-4a41-80d3-ecdd719d34b3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.375316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e4c53-6b09-4a41-80d3-ecdd719d34b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.379561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89712a4-16ad-45a1-bf9b-8f920d9e8c7d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.384115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50ef15-09fc-4493-98d2-5f66fcd5cc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.384704 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50ef15-09fc-4493-98d2-5f66fcd5cc4e admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.384979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50ef15-09fc-4493-98d2-5f66fcd5cc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.385244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50ef15-09fc-4493-98d2-5f66fcd5cc4e admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.385856 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50ef15-09fc-4493-98d2-5f66fcd5cc4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.387745 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50ef15-09fc-4493-98d2-5f66fcd5cc4e admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.394264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-357e4c53-6b09-4a41-80d3-ecdd719d34b3 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:07.395861 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-357e4c53-6b09-4a41-80d3-ecdd719d34b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:07.405277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 798/1584] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:07.406046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e4c53-6b09-4a41-80d3-ecdd719d34b3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.416224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 787/1585] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:07 2022] POST /identity/v3/users => generated 279 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:07.416803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e4c53-6b09-4a41-80d3-ecdd719d34b3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.417394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50ef15-09fc-4493-98d2-5f66fcd5cc4e admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.422547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dd7c1-159a-4220-ae20-74892c67c826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.423376 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dd7c1-159a-4220-ae20-74892c67c826 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.423651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dd7c1-159a-4220-ae20-74892c67c826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.424533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dd7c1-159a-4220-ae20-74892c67c826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.425201 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601dd7c1-159a-4220-ae20-74892c67c826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.427028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601dd7c1-159a-4220-ae20-74892c67c826 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.427745 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50ef15-09fc-4493-98d2-5f66fcd5cc4e admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.432464 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aeb362-d058-4814-bd11-54f878f99388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.433076 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeb362-d058-4814-bd11-54f878f99388 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.433346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeb362-d058-4814-bd11-54f878f99388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.433608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeb362-d058-4814-bd11-54f878f99388 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.434221 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aeb362-d058-4814-bd11-54f878f99388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.436404 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aeb362-d058-4814-bd11-54f878f99388 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.437505 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 799/1586] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:07.438278 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dd7c1-159a-4220-ae20-74892c67c826 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.447917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dd7c1-159a-4220-ae20-74892c67c826 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.452901 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f1f1d-2fb3-4995-b499-dfd20ce0b766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.453515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f1f1d-2fb3-4995-b499-dfd20ce0b766 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.453820 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f1f1d-2fb3-4995-b499-dfd20ce0b766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.454039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f1f1d-2fb3-4995-b499-dfd20ce0b766 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.455040 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f1f1d-2fb3-4995-b499-dfd20ce0b766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.456971 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f1f1d-2fb3-4995-b499-dfd20ce0b766 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.460613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 788/1587] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:07 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:07.461847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e85f7-cec8-4a6a-84d6-49dca4ce9a36 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.462120 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e85f7-cec8-4a6a-84d6-49dca4ce9a36 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.462398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e85f7-cec8-4a6a-84d6-49dca4ce9a36 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.463795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 789/1588] 10.222.0.62 () {54 vars in 962 bytes} [Thu Sep 1 16:36:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:36:07.464347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e85f7-cec8-4a6a-84d6-49dca4ce9a36 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.478196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e85f7-cec8-4a6a-84d6-49dca4ce9a36 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.480515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 800/1589] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:07 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:07.481226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f1f1d-2fb3-4995-b499-dfd20ce0b766 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.486398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c902acf-431c-455a-81fd-fb86d0f62f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.486997 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c902acf-431c-455a-81fd-fb86d0f62f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.487268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c902acf-431c-455a-81fd-fb86d0f62f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.487531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c902acf-431c-455a-81fd-fb86d0f62f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.488238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c902acf-431c-455a-81fd-fb86d0f62f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.489836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c902acf-431c-455a-81fd-fb86d0f62f67 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.490847 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f1f1d-2fb3-4995-b499-dfd20ce0b766 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.496232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c949a0a-a986-40b6-ba48-e8c92ec5e709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.496851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c949a0a-a986-40b6-ba48-e8c92ec5e709 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.497135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c949a0a-a986-40b6-ba48-e8c92ec5e709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.498674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c949a0a-a986-40b6-ba48-e8c92ec5e709 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.499322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c949a0a-a986-40b6-ba48-e8c92ec5e709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.503825 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 790/1590] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:07.504042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c949a0a-a986-40b6-ba48-e8c92ec5e709 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.504131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c902acf-431c-455a-81fd-fb86d0f62f67 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.511506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 801/1591] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:07.512437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c949a0a-a986-40b6-ba48-e8c92ec5e709 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.523353 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c949a0a-a986-40b6-ba48-e8c92ec5e709 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.527971 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51c67b-0578-473f-8501-56d06f27c61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.528702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51c67b-0578-473f-8501-56d06f27c61b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.529493 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51c67b-0578-473f-8501-56d06f27c61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.529774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51c67b-0578-473f-8501-56d06f27c61b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.530415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51c67b-0578-473f-8501-56d06f27c61b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.531997 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51c67b-0578-473f-8501-56d06f27c61b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.543083 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 802/1592] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:07.543083 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51c67b-0578-473f-8501-56d06f27c61b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.554952 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51c67b-0578-473f-8501-56d06f27c61b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.558479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c902acf-431c-455a-81fd-fb86d0f62f67 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.559632 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f681163-c0f4-43ee-aa5d-c5716d919e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.560456 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f681163-c0f4-43ee-aa5d-c5716d919e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.560753 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f681163-c0f4-43ee-aa5d-c5716d919e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.561021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f681163-c0f4-43ee-aa5d-c5716d919e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.561679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f681163-c0f4-43ee-aa5d-c5716d919e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.562358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe44a7-b751-4707-95b0-fe3cd9262135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.562959 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe44a7-b751-4707-95b0-fe3cd9262135 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.563229 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe44a7-b751-4707-95b0-fe3cd9262135 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.563339 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f681163-c0f4-43ee-aa5d-c5716d919e16 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.563493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe44a7-b751-4707-95b0-fe3cd9262135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.573119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe44a7-b751-4707-95b0-fe3cd9262135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.573274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 803/1593] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:07.573985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f681163-c0f4-43ee-aa5d-c5716d919e16 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.574521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe44a7-b751-4707-95b0-fe3cd9262135 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.584666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f681163-c0f4-43ee-aa5d-c5716d919e16 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.585188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 791/1594] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:07.586498 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c75bc9-4d5e-471c-b610-627c5edf6d78 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.586756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c75bc9-4d5e-471c-b610-627c5edf6d78 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.587029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c75bc9-4d5e-471c-b610-627c5edf6d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.589619 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758aaa8-d582-409d-9f57-1f1507103f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.590207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758aaa8-d582-409d-9f57-1f1507103f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.590486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758aaa8-d582-409d-9f57-1f1507103f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.590752 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758aaa8-d582-409d-9f57-1f1507103f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.592098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758aaa8-d582-409d-9f57-1f1507103f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.593667 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758aaa8-d582-409d-9f57-1f1507103f6c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.605753 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 804/1595] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:07.606594 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758aaa8-d582-409d-9f57-1f1507103f6c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.617159 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758aaa8-d582-409d-9f57-1f1507103f6c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.621931 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbb067-fb8e-42bc-9162-6c48bba0e3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.622731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbb067-fb8e-42bc-9162-6c48bba0e3fd admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.623137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbb067-fb8e-42bc-9162-6c48bba0e3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.623413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbb067-fb8e-42bc-9162-6c48bba0e3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.624194 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbb067-fb8e-42bc-9162-6c48bba0e3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.625793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbb067-fb8e-42bc-9162-6c48bba0e3fd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.633485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e0c75bc9-4d5e-471c-b610-627c5edf6d78 None None] MFA Rules not processed for user `2912c02ff6db4cbc97c80cd923238136`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:07.634785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e0c75bc9-4d5e-471c-b610-627c5edf6d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:07.635922 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 805/1596] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:07.636740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbb067-fb8e-42bc-9162-6c48bba0e3fd admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.647767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbb067-fb8e-42bc-9162-6c48bba0e3fd admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.653041 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360e8a9-d040-45fb-a536-2075f6b6ef96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.653817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360e8a9-d040-45fb-a536-2075f6b6ef96 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.654193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360e8a9-d040-45fb-a536-2075f6b6ef96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.654796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360e8a9-d040-45fb-a536-2075f6b6ef96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.655549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9360e8a9-d040-45fb-a536-2075f6b6ef96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.657312 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9360e8a9-d040-45fb-a536-2075f6b6ef96 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.671400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 806/1597] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:07.672909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360e8a9-d040-45fb-a536-2075f6b6ef96 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.687335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360e8a9-d040-45fb-a536-2075f6b6ef96 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.694842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa7182-645f-4d3c-b9e1-6cb4b2d9e6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.696254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa7182-645f-4d3c-b9e1-6cb4b2d9e6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.696700 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa7182-645f-4d3c-b9e1-6cb4b2d9e6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.697446 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa7182-645f-4d3c-b9e1-6cb4b2d9e6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.698485 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa7182-645f-4d3c-b9e1-6cb4b2d9e6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.701328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa7182-645f-4d3c-b9e1-6cb4b2d9e6d2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.713468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 792/1598] 10.222.0.62 () {58 vars in 1059 bytes} [Thu Sep 1 16:36:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:07.714351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c75bc9-4d5e-471c-b610-627c5edf6d78 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.714762 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 807/1599] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:07.715602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa7182-645f-4d3c-b9e1-6cb4b2d9e6d2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.726945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa7182-645f-4d3c-b9e1-6cb4b2d9e6d2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.729599 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c75bc9-4d5e-471c-b610-627c5edf6d78 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.732013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4bb1c-458b-46c1-95dc-bb6fdef77767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.733051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4bb1c-458b-46c1-95dc-bb6fdef77767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.733051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4bb1c-458b-46c1-95dc-bb6fdef77767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.733268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4bb1c-458b-46c1-95dc-bb6fdef77767 admin admin] PATH_INFO: `/v3/projects/af2ea34aea99498eb36647be71aec5e8/users/2c1c3f8f28b147fc838533baf62b72af/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.735535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4ef06-5457-414e-87d3-ff77517dabdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.736123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4ef06-5457-414e-87d3-ff77517dabdb admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.736437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4ef06-5457-414e-87d3-ff77517dabdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.737699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4ef06-5457-414e-87d3-ff77517dabdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.737699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4ef06-5457-414e-87d3-ff77517dabdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.739311 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4ef06-5457-414e-87d3-ff77517dabdb admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.743759 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:07.752146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4bb1c-458b-46c1-95dc-bb6fdef77767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2ea34aea99498eb36647be71aec5e8, user_id=2c1c3f8f28b147fc838533baf62b72af, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.755164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4bb1c-458b-46c1-95dc-bb6fdef77767 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.759280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 793/1600] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:07.759280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4ef06-5457-414e-87d3-ff77517dabdb admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.771353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4ef06-5457-414e-87d3-ff77517dabdb admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.778125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 808/1601] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:07 2022] PUT /identity/v3/projects/af2ea34aea99498eb36647be71aec5e8/users/2c1c3f8f28b147fc838533baf62b72af/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:07.779094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4bb1c-458b-46c1-95dc-bb6fdef77767 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.831583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:07.840528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be889f-b972-47c6-a82d-5a9349836cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.841247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be889f-b972-47c6-a82d-5a9349836cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.841517 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be889f-b972-47c6-a82d-5a9349836cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.841840 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be889f-b972-47c6-a82d-5a9349836cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.842635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be889f-b972-47c6-a82d-5a9349836cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.844353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be889f-b972-47c6-a82d-5a9349836cf8 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.851012 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4bb1c-458b-46c1-95dc-bb6fdef77767 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.855643 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 794/1602] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:07.856514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18bd22-66c3-4143-a98f-7b2b1d2e7a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.856640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be889f-b972-47c6-a82d-5a9349836cf8 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.857859 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18bd22-66c3-4143-a98f-7b2b1d2e7a19 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.858483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18bd22-66c3-4143-a98f-7b2b1d2e7a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.859088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18bd22-66c3-4143-a98f-7b2b1d2e7a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.860543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b18bd22-66c3-4143-a98f-7b2b1d2e7a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.864497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b18bd22-66c3-4143-a98f-7b2b1d2e7a19 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.876260 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be889f-b972-47c6-a82d-5a9349836cf8 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.878075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 809/1603] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:07.878984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18bd22-66c3-4143-a98f-7b2b1d2e7a19 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.889237 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c33e44-49f4-4b4f-b740-c7aa45074858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.891401 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c33e44-49f4-4b4f-b740-c7aa45074858 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.891401 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c33e44-49f4-4b4f-b740-c7aa45074858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.891624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18bd22-66c3-4143-a98f-7b2b1d2e7a19 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:07.891730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c33e44-49f4-4b4f-b740-c7aa45074858 admin admin] PATH_INFO: `/v3/projects/601c248b5d9b48488351cb4d9561a2f4/users/52760b6736e14ccd8df8aca0015cb292/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.896193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4b7ab-f70f-493e-8da9-a8065c6acaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:07.896930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4b7ab-f70f-493e-8da9-a8065c6acaff admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:07.897214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4b7ab-f70f-493e-8da9-a8065c6acaff admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:07.897488 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4b7ab-f70f-493e-8da9-a8065c6acaff admin admin] PATH_INFO: `/v3/projects/c6eb3ba21eae45898f42f3456591591e/users/3a9b222396b1438f9fa0af0e197a9293/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:07.908774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c33e44-49f4-4b4f-b740-c7aa45074858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=601c248b5d9b48488351cb4d9561a2f4, user_id=52760b6736e14ccd8df8aca0015cb292, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.911644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c33e44-49f4-4b4f-b740-c7aa45074858 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.913647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4b7ab-f70f-493e-8da9-a8065c6acaff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6eb3ba21eae45898f42f3456591591e, user_id=3a9b222396b1438f9fa0af0e197a9293, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:07.916339 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4b7ab-f70f-493e-8da9-a8065c6acaff admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:07.928421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 795/1604] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:07 2022] PUT /identity/v3/projects/601c248b5d9b48488351cb4d9561a2f4/users/52760b6736e14ccd8df8aca0015cb292/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:07.929365 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c33e44-49f4-4b4f-b740-c7aa45074858 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:07.936331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 810/1605] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:07 2022] PUT /identity/v3/projects/c6eb3ba21eae45898f42f3456591591e/users/3a9b222396b1438f9fa0af0e197a9293/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:07.937818 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4b7ab-f70f-493e-8da9-a8065c6acaff admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:08.009787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4b7ab-f70f-493e-8da9-a8065c6acaff admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:08.011610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c33e44-49f4-4b4f-b740-c7aa45074858 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:08.015194 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b723a42-bef9-4340-921f-5213069ff28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:08.015919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b723a42-bef9-4340-921f-5213069ff28c admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.016265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b723a42-bef9-4340-921f-5213069ff28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.016545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b723a42-bef9-4340-921f-5213069ff28c admin admin] PATH_INFO: `/v3/projects/2ac77723842947a5b70a97ad0701711b/users/7b20bf54d9a94e62a119026861dacd2a/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.034715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b723a42-bef9-4340-921f-5213069ff28c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac77723842947a5b70a97ad0701711b, user_id=7b20bf54d9a94e62a119026861dacd2a, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:08.037449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b723a42-bef9-4340-921f-5213069ff28c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:08.058092 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 811/1606] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:07 2022] PUT /identity/v3/projects/2ac77723842947a5b70a97ad0701711b/users/7b20bf54d9a94e62a119026861dacd2a/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:08.060050 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b8967-72a6-4f65-898f-08346e0c91da None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.060349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b8967-72a6-4f65-898f-08346e0c91da None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.060684 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b8967-72a6-4f65-898f-08346e0c91da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.090835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-484b8967-72a6-4f65-898f-08346e0c91da None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:08.092461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-484b8967-72a6-4f65-898f-08346e0c91da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:08.103736 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f46634-6889-4bca-ba8f-517a532e95cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:08.104543 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f46634-6889-4bca-ba8f-517a532e95cc service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.105053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f46634-6889-4bca-ba8f-517a532e95cc service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.105489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f46634-6889-4bca-ba8f-517a532e95cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.111190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:08.157332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 812/1607] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:08.158166 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b8967-72a6-4f65-898f-08346e0c91da None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:08.172614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b8967-72a6-4f65-898f-08346e0c91da None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:08.176143 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f46634-6889-4bca-ba8f-517a532e95cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:08.178043 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f46634-6889-4bca-ba8f-517a532e95cc service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:08.179066 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9928df4-dc86-4d4e-9dec-9fb2ab761881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:08.179908 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9928df4-dc86-4d4e-9dec-9fb2ab761881 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.180224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9928df4-dc86-4d4e-9dec-9fb2ab761881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.180547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9928df4-dc86-4d4e-9dec-9fb2ab761881 admin admin] PATH_INFO: `/v3/projects/130966aabcc04fc98c9f5139f826e78f/users/f21b1fea950d48ef9a55cded8ed172a6/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.188353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 796/1608] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:08.189527 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f46634-6889-4bca-ba8f-517a532e95cc service nova] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:08.200388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f46634-6889-4bca-ba8f-517a532e95cc service nova] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:08.206620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeab105-d57b-4294-b0c3-928e59906221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:08.207131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeab105-d57b-4294-b0c3-928e59906221 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.207474 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeab105-d57b-4294-b0c3-928e59906221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.207808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeab105-d57b-4294-b0c3-928e59906221 admin admin] PATH_INFO: `/v3/projects/b7ebaf9d1f494b7ab9f5adde08010a4d/users/33d52af42954466e991986fd550e9270/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.217322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9928df4-dc86-4d4e-9dec-9fb2ab761881 admin admin] RBAC: Authorizing `identity:create_grant(project_id=130966aabcc04fc98c9f5139f826e78f, user_id=f21b1fea950d48ef9a55cded8ed172a6, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:08.222149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9928df4-dc86-4d4e-9dec-9fb2ab761881 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:08.228833 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeab105-d57b-4294-b0c3-928e59906221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ebaf9d1f494b7ab9f5adde08010a4d, user_id=33d52af42954466e991986fd550e9270, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:08.232093 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeab105-d57b-4294-b0c3-928e59906221 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:08.243333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 813/1609] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:08 2022] PUT /identity/v3/projects/130966aabcc04fc98c9f5139f826e78f/users/f21b1fea950d48ef9a55cded8ed172a6/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:08.245179 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cdd87-d9ca-4828-b1e0-dc470bd66f76 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.245636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cdd87-d9ca-4828-b1e0-dc470bd66f76 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.246073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cdd87-d9ca-4828-b1e0-dc470bd66f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.255984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 797/1610] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:08 2022] PUT /identity/v3/projects/b7ebaf9d1f494b7ab9f5adde08010a4d/users/33d52af42954466e991986fd550e9270/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:08.257730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bd169-47df-4b6a-828a-72b268b53631 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.258243 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bd169-47df-4b6a-828a-72b268b53631 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.258680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bd169-47df-4b6a-828a-72b268b53631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.285148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e92cdd87-d9ca-4828-b1e0-dc470bd66f76 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:08.286970 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e92cdd87-d9ca-4828-b1e0-dc470bd66f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:08.299161 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e11bd169-47df-4b6a-828a-72b268b53631 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:08.300563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e11bd169-47df-4b6a-828a-72b268b53631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:08.307668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:08.376519 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 798/1611] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:08.378548 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff047c-dc76-4b38-bdda-898986af3c50 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.379058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff047c-dc76-4b38-bdda-898986af3c50 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.379487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff047c-dc76-4b38-bdda-898986af3c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.391896 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 814/1612] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:08.393071 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cdd87-d9ca-4828-b1e0-dc470bd66f76 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:08.427582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-b4ff047c-dc76-4b38-bdda-898986af3c50 None None] MFA Rules not processed for user `7b20bf54d9a94e62a119026861dacd2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:08.431200 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b4ff047c-dc76-4b38-bdda-898986af3c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:08.463281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cdd87-d9ca-4828-b1e0-dc470bd66f76 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:08.468049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fdc46-8b2d-4e40-9b86-85196e9e632b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:08.473833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fdc46-8b2d-4e40-9b86-85196e9e632b service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.474143 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fdc46-8b2d-4e40-9b86-85196e9e632b service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.474494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fdc46-8b2d-4e40-9b86-85196e9e632b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.486616 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fdc46-8b2d-4e40-9b86-85196e9e632b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:08.488443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fdc46-8b2d-4e40-9b86-85196e9e632b service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:08.501442 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 815/1613] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:08.502730 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a13e9-86cf-4627-a3b9-97cecc6a0401 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.503023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a13e9-86cf-4627-a3b9-97cecc6a0401 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.503283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a13e9-86cf-4627-a3b9-97cecc6a0401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.522166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 799/1614] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:08 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:08.523995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2667f36-ce4d-4d20-aa68-9fba1fe60872 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.524389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2667f36-ce4d-4d20-aa68-9fba1fe60872 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.524798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2667f36-ce4d-4d20-aa68-9fba1fe60872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.555877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cf8a13e9-86cf-4627-a3b9-97cecc6a0401 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:08.556899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:08.557019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cf8a13e9-86cf-4627-a3b9-97cecc6a0401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:08.571777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 816/1615] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:08.572146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a13e9-86cf-4627-a3b9-97cecc6a0401 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:08.579752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-b2667f36-ce4d-4d20-aa68-9fba1fe60872 None None] MFA Rules not processed for user `33d52af42954466e991986fd550e9270`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:08.583773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b2667f36-ce4d-4d20-aa68-9fba1fe60872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:08.596113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a13e9-86cf-4627-a3b9-97cecc6a0401 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:08.603075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881018b-96d5-433a-aef6-a6d672e1ac68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:08.603794 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881018b-96d5-433a-aef6-a6d672e1ac68 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.604113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881018b-96d5-433a-aef6-a6d672e1ac68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.604357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881018b-96d5-433a-aef6-a6d672e1ac68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.628075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881018b-96d5-433a-aef6-a6d672e1ac68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:08.629841 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881018b-96d5-433a-aef6-a6d672e1ac68 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:08.651709 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 817/1616] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:08.655533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881018b-96d5-433a-aef6-a6d672e1ac68 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:08.676727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881018b-96d5-433a-aef6-a6d672e1ac68 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:08.685261 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bdc11-12ad-4dbc-8d82-c1fd7fb2a373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:08.685910 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bdc11-12ad-4dbc-8d82-c1fd7fb2a373 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.686328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bdc11-12ad-4dbc-8d82-c1fd7fb2a373 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.686650 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bdc11-12ad-4dbc-8d82-c1fd7fb2a373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.699225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bdc11-12ad-4dbc-8d82-c1fd7fb2a373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:08.701932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bdc11-12ad-4dbc-8d82-c1fd7fb2a373 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:08.715183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 818/1617] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:08.717141 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73eeeb-e202-476b-9e78-99c73d193be3 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.717472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73eeeb-e202-476b-9e78-99c73d193be3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.717750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73eeeb-e202-476b-9e78-99c73d193be3 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.719432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 819/1618] 10.222.0.62 () {54 vars in 912 bytes} [Thu Sep 1 16:36:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:36:08.720536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73eeeb-e202-476b-9e78-99c73d193be3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:08.727817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 800/1619] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:08.729644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2667f36-ce4d-4d20-aa68-9fba1fe60872 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:08.741795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2667f36-ce4d-4d20-aa68-9fba1fe60872 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:08.745993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e5355-1535-4b37-a9df-52d4e805645d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:08.746843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e5355-1535-4b37-a9df-52d4e805645d service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.747122 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e5355-1535-4b37-a9df-52d4e805645d service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.747514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e5355-1535-4b37-a9df-52d4e805645d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.757816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73eeeb-e202-476b-9e78-99c73d193be3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:08.768261 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebb8c6-0557-4929-b7cf-10bb5066cc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:08.769060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebb8c6-0557-4929-b7cf-10bb5066cc39 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.769337 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebb8c6-0557-4929-b7cf-10bb5066cc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.769647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebb8c6-0557-4929-b7cf-10bb5066cc39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.770878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebb8c6-0557-4929-b7cf-10bb5066cc39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:08.773576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e5355-1535-4b37-a9df-52d4e805645d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:08.773809 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebb8c6-0557-4929-b7cf-10bb5066cc39 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:08.776136 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e5355-1535-4b37-a9df-52d4e805645d service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:08.791506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 801/1620] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:08.794676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a964d6-6b10-427b-89fe-bd884b9bbe4e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.795113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a964d6-6b10-427b-89fe-bd884b9bbe4e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.795410 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a964d6-6b10-427b-89fe-bd884b9bbe4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.830094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-c8a964d6-6b10-427b-89fe-bd884b9bbe4e None None] MFA Rules not processed for user `2912c02ff6db4cbc97c80cd923238136`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:08.831307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c8a964d6-6b10-427b-89fe-bd884b9bbe4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:08.843555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 820/1621] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:08 2022] POST /identity/v3/projects => generated 357 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:08.845168 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e793d4-1215-44aa-af2d-d096e9c7628f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.845533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e793d4-1215-44aa-af2d-d096e9c7628f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.845805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e793d4-1215-44aa-af2d-d096e9c7628f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.847448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 821/1622] 10.222.0.62 () {54 vars in 962 bytes} [Thu Sep 1 16:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:36:08.848184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e793d4-1215-44aa-af2d-d096e9c7628f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:08.939313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e793d4-1215-44aa-af2d-d096e9c7628f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:08.945921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72186c-769f-4265-8f53-74670014dae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:08.946912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72186c-769f-4265-8f53-74670014dae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.947487 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72186c-769f-4265-8f53-74670014dae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.948146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72186c-769f-4265-8f53-74670014dae9 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.949257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72186c-769f-4265-8f53-74670014dae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:08.952005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72186c-769f-4265-8f53-74670014dae9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:08.964567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:08.981381 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 822/1623] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:08 2022] POST /identity/v3/users => generated 285 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:08.983430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66ea6f-906c-4f29-a37b-9df4ac1bc525 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:08.983717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66ea6f-906c-4f29-a37b-9df4ac1bc525 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:08.984234 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66ea6f-906c-4f29-a37b-9df4ac1bc525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:08.987337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 802/1624] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:36:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:08.988561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a964d6-6b10-427b-89fe-bd884b9bbe4e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:09.002947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a964d6-6b10-427b-89fe-bd884b9bbe4e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:09.008727 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde180f1-e9a0-4481-b3a6-850d8245ca6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:09.009542 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde180f1-e9a0-4481-b3a6-850d8245ca6d admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:09.009939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde180f1-e9a0-4481-b3a6-850d8245ca6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:09.010649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde180f1-e9a0-4481-b3a6-850d8245ca6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:09.011273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde180f1-e9a0-4481-b3a6-850d8245ca6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:09.013436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde180f1-e9a0-4481-b3a6-850d8245ca6d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:09.017915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0b66ea6f-906c-4f29-a37b-9df4ac1bc525 None None] MFA Rules not processed for user `2912c02ff6db4cbc97c80cd923238136`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:09.019736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0b66ea6f-906c-4f29-a37b-9df4ac1bc525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:09.026433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 803/1625] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:09.027516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde180f1-e9a0-4481-b3a6-850d8245ca6d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:09.140695 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 823/1626] 10.222.0.62 () {58 vars in 1059 bytes} [Thu Sep 1 16:36:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:09.141921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66ea6f-906c-4f29-a37b-9df4ac1bc525 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:09.152174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde180f1-e9a0-4481-b3a6-850d8245ca6d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:09.156411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66ea6f-906c-4f29-a37b-9df4ac1bc525 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:09.161515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ba8a7-57d7-4e0c-be93-23ba02ca9d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:09.162206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ba8a7-57d7-4e0c-be93-23ba02ca9d85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:09.162484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ba8a7-57d7-4e0c-be93-23ba02ca9d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:09.162753 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ba8a7-57d7-4e0c-be93-23ba02ca9d85 admin admin] PATH_INFO: `/v3/projects/62e10b2106c64dcdbeb4bc50294515ec/users/05db1b00305f4d088ffa6efd69774e43/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:09.166821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0c668-dd7a-461d-bacc-f2d8b1478b83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:09.168361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0c668-dd7a-461d-bacc-f2d8b1478b83 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:09.168361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0c668-dd7a-461d-bacc-f2d8b1478b83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:09.168599 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0c668-dd7a-461d-bacc-f2d8b1478b83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:09.181808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ba8a7-57d7-4e0c-be93-23ba02ca9d85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62e10b2106c64dcdbeb4bc50294515ec, user_id=05db1b00305f4d088ffa6efd69774e43, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:09.185297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ba8a7-57d7-4e0c-be93-23ba02ca9d85 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:09.188959 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0c668-dd7a-461d-bacc-f2d8b1478b83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:09.192568 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0c668-dd7a-461d-bacc-f2d8b1478b83 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:09.211208 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 824/1627] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:09 2022] PUT /identity/v3/projects/62e10b2106c64dcdbeb4bc50294515ec/users/05db1b00305f4d088ffa6efd69774e43/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:09.212193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ba8a7-57d7-4e0c-be93-23ba02ca9d85 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:09.232410 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 804/1628] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:09.233651 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0c668-dd7a-461d-bacc-f2d8b1478b83 service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:09.259052 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0c668-dd7a-461d-bacc-f2d8b1478b83 service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:09.280566 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:09.395607 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e6437-12e7-413b-903b-eafe30c849ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:09.396400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e6437-12e7-413b-903b-eafe30c849ca admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:09.396683 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e6437-12e7-413b-903b-eafe30c849ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:09.396982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ba8a7-57d7-4e0c-be93-23ba02ca9d85 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:09.397113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e6437-12e7-413b-903b-eafe30c849ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:09.397632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e6437-12e7-413b-903b-eafe30c849ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:09.402445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e6437-12e7-413b-903b-eafe30c849ca admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:09.405731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ceae3-df3d-4c11-9c85-4a06c5c058d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:09.406620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ceae3-df3d-4c11-9c85-4a06c5c058d8 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:09.407089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ceae3-df3d-4c11-9c85-4a06c5c058d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:09.407446 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ceae3-df3d-4c11-9c85-4a06c5c058d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:09.428652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 805/1629] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:09.433127 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e6437-12e7-413b-903b-eafe30c849ca admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:09.454867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e6437-12e7-413b-903b-eafe30c849ca admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:09.464980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23637ee7-10f6-4ed6-8bed-262e355519b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:09.465786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23637ee7-10f6-4ed6-8bed-262e355519b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:09.466248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23637ee7-10f6-4ed6-8bed-262e355519b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:09.466546 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23637ee7-10f6-4ed6-8bed-262e355519b6 admin admin] PATH_INFO: `/v3/projects/62e10b2106c64dcdbeb4bc50294515ec/users/05db1b00305f4d088ffa6efd69774e43/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:09.472186 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23637ee7-10f6-4ed6-8bed-262e355519b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62e10b2106c64dcdbeb4bc50294515ec, user_id=05db1b00305f4d088ffa6efd69774e43, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:09.474900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23637ee7-10f6-4ed6-8bed-262e355519b6 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:09.503784 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 806/1630] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:09 2022] PUT /identity/v3/projects/62e10b2106c64dcdbeb4bc50294515ec/users/05db1b00305f4d088ffa6efd69774e43/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:09.514747 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051766b0-c877-41e9-aac4-aaff44d3730a None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:09.515084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051766b0-c877-41e9-aac4-aaff44d3730a None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:09.515389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051766b0-c877-41e9-aac4-aaff44d3730a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:09.531872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ceae3-df3d-4c11-9c85-4a06c5c058d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:09.535362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ceae3-df3d-4c11-9c85-4a06c5c058d8 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:09.542016 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:09.568454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 825/1631] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:09.620321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-051766b0-c877-41e9-aac4-aaff44d3730a None None] MFA Rules not processed for user `05db1b00305f4d088ffa6efd69774e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:09.621580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-051766b0-c877-41e9-aac4-aaff44d3730a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:09.834306 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 807/1632] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:09 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:09.844889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ceae3-df3d-4c11-9c85-4a06c5c058d8 service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:10.012565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ceae3-df3d-4c11-9c85-4a06c5c058d8 service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:10.141750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce7a57-256b-46e9-aa12-b74e64270583 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:10.144888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce7a57-256b-46e9-aa12-b74e64270583 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:10.145159 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce7a57-256b-46e9-aa12-b74e64270583 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:10.145439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce7a57-256b-46e9-aa12-b74e64270583 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:10.176151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce7a57-256b-46e9-aa12-b74e64270583 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:10.179223 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b8cf4-5295-4ac1-9622-25319760e657 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:10.179711 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b8cf4-5295-4ac1-9622-25319760e657 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:10.179909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b8cf4-5295-4ac1-9622-25319760e657 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:10.181230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 808/1633] 10.222.0.62 () {54 vars in 912 bytes} [Thu Sep 1 16:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:36:10.181358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce7a57-256b-46e9-aa12-b74e64270583 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:10.189099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326a5a5-13eb-4de0-8685-d941c51d63bd None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:10.189378 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326a5a5-13eb-4de0-8685-d941c51d63bd None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:10.189649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326a5a5-13eb-4de0-8685-d941c51d63bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:10.214454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 826/1634] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:09 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:10.232814 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-b326a5a5-13eb-4de0-8685-d941c51d63bd None None] MFA Rules not processed for user `2912c02ff6db4cbc97c80cd923238136`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:10.233986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b326a5a5-13eb-4de0-8685-d941c51d63bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:10.432214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 809/1635] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:36:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:10.478801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce7a57-256b-46e9-aa12-b74e64270583 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:10.495448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce7a57-256b-46e9-aa12-b74e64270583 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:10.500024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb5d34-9e59-4621-8416-88a962a2849f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:10.500654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5d34-9e59-4621-8416-88a962a2849f service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:10.500939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5d34-9e59-4621-8416-88a962a2849f service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:10.501181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5d34-9e59-4621-8416-88a962a2849f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:10.533105 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb5d34-9e59-4621-8416-88a962a2849f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:10.539470 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb5d34-9e59-4621-8416-88a962a2849f service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:10.557480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 827/1636] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:10.598678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326a5a5-13eb-4de0-8685-d941c51d63bd None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:10.610919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326a5a5-13eb-4de0-8685-d941c51d63bd None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:10.615328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a724dcc-d9b8-43b7-ac1e-cbd16b269d4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:10.616284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a724dcc-d9b8-43b7-ac1e-cbd16b269d4a service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:10.616854 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a724dcc-d9b8-43b7-ac1e-cbd16b269d4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:10.617406 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a724dcc-d9b8-43b7-ac1e-cbd16b269d4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:10.628395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:10.692521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a724dcc-d9b8-43b7-ac1e-cbd16b269d4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:10.694816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a724dcc-d9b8-43b7-ac1e-cbd16b269d4a service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:10.707809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 810/1637] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:10 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:16.698857 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb5d34-9e59-4621-8416-88a962a2849f service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:16.897427 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb5d34-9e59-4621-8416-88a962a2849f service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:16.908329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e40df-6a5d-469a-84a9-92e21c4c982e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:16.909010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e40df-6a5d-469a-84a9-92e21c4c982e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:16.909275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e40df-6a5d-469a-84a9-92e21c4c982e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:16.909538 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e40df-6a5d-469a-84a9-92e21c4c982e admin admin] PATH_INFO: `/v3/users/33d52af42954466e991986fd550e9270` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:16.916459 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e40df-6a5d-469a-84a9-92e21c4c982e admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d52af42954466e991986fd550e9270)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:16.918821 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e40df-6a5d-469a-84a9-92e21c4c982e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:17.044490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:17.055036 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c7e40df-6a5d-469a-84a9-92e21c4c982e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d52af42954466e991986fd550e9270'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:17.090288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c7e40df-6a5d-469a-84a9-92e21c4c982e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d52af42954466e991986fd550e9270'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:17.095929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c7e40df-6a5d-469a-84a9-92e21c4c982e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d52af42954466e991986fd550e9270'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:17.152224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c7e40df-6a5d-469a-84a9-92e21c4c982e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d52af42954466e991986fd550e9270'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:17.196829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c7e40df-6a5d-469a-84a9-92e21c4c982e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d52af42954466e991986fd550e9270'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:17.213830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 828/1638] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:16 2022] DELETE /identity/v3/users/33d52af42954466e991986fd550e9270 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:17.215296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f1486-8357-4cd9-a8c9-7de6e5c5ea3b None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:17.215882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f1486-8357-4cd9-a8c9-7de6e5c5ea3b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:17.215999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f1486-8357-4cd9-a8c9-7de6e5c5ea3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:17.263370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-fa9f1486-8357-4cd9-a8c9-7de6e5c5ea3b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:17.294271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-fa9f1486-8357-4cd9-a8c9-7de6e5c5ea3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:17.505586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 811/1639] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:17.555412 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e40df-6a5d-469a-84a9-92e21c4c982e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:17.556870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4c7e40df-6a5d-469a-84a9-92e21c4c982e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:17.765149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e40df-6a5d-469a-84a9-92e21c4c982e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:17.778822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42ae73-9f6c-457b-bcfd-7b0ff33cd9bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:17.779458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42ae73-9f6c-457b-bcfd-7b0ff33cd9bd service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:17.779729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42ae73-9f6c-457b-bcfd-7b0ff33cd9bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:17.780018 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42ae73-9f6c-457b-bcfd-7b0ff33cd9bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:17.813148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42ae73-9f6c-457b-bcfd-7b0ff33cd9bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:17.814680 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42ae73-9f6c-457b-bcfd-7b0ff33cd9bd service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:17.853848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 829/1640] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:19.247686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f1486-8357-4cd9-a8c9-7de6e5c5ea3b None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:19.249402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-fa9f1486-8357-4cd9-a8c9-7de6e5c5ea3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:19.262480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f1486-8357-4cd9-a8c9-7de6e5c5ea3b None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:19.267778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:19.271245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:19.271699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:19.272141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] PATH_INFO: `/v3/users/f1c0a7b17511415ea9316dc8e61dc8c9` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:19.274897 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1c0a7b17511415ea9316dc8e61dc8c9)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:19.276150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: WARNING py.warnings [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:36:19.276150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: warnings.warn(msg) Sep 01 16:36:19.276150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: Sep 01 16:36:19.277922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:19.384166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1c0a7b17511415ea9316dc8e61dc8c9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:19.410520 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1c0a7b17511415ea9316dc8e61dc8c9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:19.423731 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1c0a7b17511415ea9316dc8e61dc8c9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:19.438524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1c0a7b17511415ea9316dc8e61dc8c9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:19.439071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1c0a7b17511415ea9316dc8e61dc8c9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:19.450415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 812/1641] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:19 2022] DELETE /identity/v3/users/f1c0a7b17511415ea9316dc8e61dc8c9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:19.455802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126215f-0274-4a6c-befb-a14fcf71dbfe None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:19.456362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126215f-0274-4a6c-befb-a14fcf71dbfe None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:19.456503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126215f-0274-4a6c-befb-a14fcf71dbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:19.494252 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2126215f-0274-4a6c-befb-a14fcf71dbfe None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:19.500124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2126215f-0274-4a6c-befb-a14fcf71dbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:19.719448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 830/1642] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:19.854595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:19.857986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:19.899054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:19.954735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e7b32-b403-4103-9b6a-685065a6220e admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:19.959921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc046ef-deed-43a7-9e3b-8126ba20e71e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:19.960659 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc046ef-deed-43a7-9e3b-8126ba20e71e service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:19.960940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc046ef-deed-43a7-9e3b-8126ba20e71e service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:19.962478 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc046ef-deed-43a7-9e3b-8126ba20e71e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:19.974782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc046ef-deed-43a7-9e3b-8126ba20e71e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:19.976927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc046ef-deed-43a7-9e3b-8126ba20e71e service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:19.989353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 813/1643] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:20.581492 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126215f-0274-4a6c-befb-a14fcf71dbfe None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:20.583041 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2126215f-0274-4a6c-befb-a14fcf71dbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:20.585209 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae9be1-82a4-4d87-bdf6-3e310ec21b15 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:20.585679 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae9be1-82a4-4d87-bdf6-3e310ec21b15 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:20.585887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae9be1-82a4-4d87-bdf6-3e310ec21b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:20.616293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126215f-0274-4a6c-befb-a14fcf71dbfe None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:20.633379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27323dc-cefb-4af8-af86-670d41b80fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:20.634254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27323dc-cefb-4af8-af86-670d41b80fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:20.634671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27323dc-cefb-4af8-af86-670d41b80fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:20.635087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27323dc-cefb-4af8-af86-670d41b80fa1 admin admin] PATH_INFO: `/v3/projects/b7ebaf9d1f494b7ab9f5adde08010a4d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:20.636705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27323dc-cefb-4af8-af86-670d41b80fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ebaf9d1f494b7ab9f5adde08010a4d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:20.646625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-d27323dc-cefb-4af8-af86-670d41b80fa1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:36:20.646625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 01 16:36:20.646625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: Sep 01 16:36:20.650776 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27323dc-cefb-4af8-af86-670d41b80fa1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:20.675151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-2eae9be1-82a4-4d87-bdf6-3e310ec21b15 None None] MFA Rules not processed for user `849f875ede78424c81919df6d9c152dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:20.680351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2eae9be1-82a4-4d87-bdf6-3e310ec21b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:20.834567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d27323dc-cefb-4af8-af86-670d41b80fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ebaf9d1f494b7ab9f5adde08010a4d', 'request_id': 'req-d27323dc-cefb-4af8-af86-670d41b80fa1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:20.854619 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d27323dc-cefb-4af8-af86-670d41b80fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ebaf9d1f494b7ab9f5adde08010a4d', 'request_id': 'req-d27323dc-cefb-4af8-af86-670d41b80fa1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:20.866392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:20.885984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d27323dc-cefb-4af8-af86-670d41b80fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ebaf9d1f494b7ab9f5adde08010a4d', 'request_id': 'req-d27323dc-cefb-4af8-af86-670d41b80fa1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:20.908486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 814/1644] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:20.924071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae9be1-82a4-4d87-bdf6-3e310ec21b15 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:20.925634 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2eae9be1-82a4-4d87-bdf6-3e310ec21b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:20.927090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d27323dc-cefb-4af8-af86-670d41b80fa1 admin admin] The token cache is being invalidate because project b7ebaf9d1f494b7ab9f5adde08010a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:20.927388 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d27323dc-cefb-4af8-af86-670d41b80fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:20.936438 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d27323dc-cefb-4af8-af86-670d41b80fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:20.946617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 831/1645] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:20 2022] DELETE /identity/v3/projects/b7ebaf9d1f494b7ab9f5adde08010a4d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:20.949448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1416b54-be90-4e31-8bd7-39c112eafe37 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:20.950649 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1416b54-be90-4e31-8bd7-39c112eafe37 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:20.951023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1416b54-be90-4e31-8bd7-39c112eafe37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:20.988336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f1416b54-be90-4e31-8bd7-39c112eafe37 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:20.989741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f1416b54-be90-4e31-8bd7-39c112eafe37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:21.100713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae9be1-82a4-4d87-bdf6-3e310ec21b15 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:21.124550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 832/1646] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:21.127232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce970ed1-f2dc-4335-86c9-50dcd9960a4f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:21.127531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce970ed1-f2dc-4335-86c9-50dcd9960a4f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:21.127810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce970ed1-f2dc-4335-86c9-50dcd9960a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:21.170811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:21.199325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ce970ed1-f2dc-4335-86c9-50dcd9960a4f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:21.199325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ce970ed1-f2dc-4335-86c9-50dcd9960a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:21.211442 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 833/1647] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:21.219981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce970ed1-f2dc-4335-86c9-50dcd9960a4f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:21.236814 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62452364-f772-4e72-9c9c-abfac4479462 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:21.237585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62452364-f772-4e72-9c9c-abfac4479462 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:21.241274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62452364-f772-4e72-9c9c-abfac4479462 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:21.241274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62452364-f772-4e72-9c9c-abfac4479462 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:21.243510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-62452364-f772-4e72-9c9c-abfac4479462 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:21.245664 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce970ed1-f2dc-4335-86c9-50dcd9960a4f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:21.251711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01ea91-c2f8-4702-a806-2e15c308ec7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:21.255539 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01ea91-c2f8-4702-a806-2e15c308ec7f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:21.256106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01ea91-c2f8-4702-a806-2e15c308ec7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:21.256544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01ea91-c2f8-4702-a806-2e15c308ec7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:21.257430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01ea91-c2f8-4702-a806-2e15c308ec7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:21.257981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01ea91-c2f8-4702-a806-2e15c308ec7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:21.260088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01ea91-c2f8-4702-a806-2e15c308ec7f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:21.284144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 834/1648] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:36:21.288213 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01ea91-c2f8-4702-a806-2e15c308ec7f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:21.328574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01ea91-c2f8-4702-a806-2e15c308ec7f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:21.344598 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63e336-9251-469f-93b3-b2356e060128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:21.345737 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63e336-9251-469f-93b3-b2356e060128 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:21.346021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63e336-9251-469f-93b3-b2356e060128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:21.346588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63e336-9251-469f-93b3-b2356e060128 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:21.346937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63e336-9251-469f-93b3-b2356e060128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:21.350030 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63e336-9251-469f-93b3-b2356e060128 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:21.428392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62452364-f772-4e72-9c9c-abfac4479462 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:21.445706 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62452364-f772-4e72-9c9c-abfac4479462 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:21.461852 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 835/1649] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:21 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:21.467043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63e336-9251-469f-93b3-b2356e060128 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:21.486304 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 815/1650] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:21.616100 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63e336-9251-469f-93b3-b2356e060128 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:21.628216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3b262-08c5-4163-8fc9-be3e6abeda7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:21.628400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3b262-08c5-4163-8fc9-be3e6abeda7d admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:21.628848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3b262-08c5-4163-8fc9-be3e6abeda7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:21.632397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3b262-08c5-4163-8fc9-be3e6abeda7d admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:21.636717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3b262-08c5-4163-8fc9-be3e6abeda7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:21.639437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3b262-08c5-4163-8fc9-be3e6abeda7d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:21.715947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 836/1651] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:21 2022] POST /identity/v3/users => generated 283 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:21.716104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62452364-f772-4e72-9c9c-abfac4479462 service nova] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:21.729782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62452364-f772-4e72-9c9c-abfac4479462 service nova] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:21.741752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f8b81-b1f5-4eba-bb59-f48e289d612a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:21.742397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f8b81-b1f5-4eba-bb59-f48e289d612a admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:21.742672 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f8b81-b1f5-4eba-bb59-f48e289d612a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:21.742940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f8b81-b1f5-4eba-bb59-f48e289d612a admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:21.743588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f8b81-b1f5-4eba-bb59-f48e289d612a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:21.745255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f8b81-b1f5-4eba-bb59-f48e289d612a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:21.777075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 816/1652] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:36:21.778865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f8b81-b1f5-4eba-bb59-f48e289d612a admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:21.779563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3b262-08c5-4163-8fc9-be3e6abeda7d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:21.784976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-21f3b262-08c5-4163-8fc9-be3e6abeda7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:21.812707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f8b81-b1f5-4eba-bb59-f48e289d612a admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:21.820654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3b262-08c5-4163-8fc9-be3e6abeda7d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:21.825810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028a7a1-c6c7-45ce-9a7c-786aa6f4479d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:21.826502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028a7a1-c6c7-45ce-9a7c-786aa6f4479d admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:21.827057 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028a7a1-c6c7-45ce-9a7c-786aa6f4479d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:21.827407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028a7a1-c6c7-45ce-9a7c-786aa6f4479d admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:21.828143 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a028a7a1-c6c7-45ce-9a7c-786aa6f4479d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:21.833616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a028a7a1-c6c7-45ce-9a7c-786aa6f4479d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:21.844518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4968b-e894-417e-ad3a-44cdc29db5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:21.846916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4968b-e894-417e-ad3a-44cdc29db5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:21.848899 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4968b-e894-417e-ad3a-44cdc29db5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:21.849419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4968b-e894-417e-ad3a-44cdc29db5f2 admin admin] PATH_INFO: `/v3/projects/276464f90d71465097dae2fe81bdf031` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:21.858202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 817/1653] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:21.861589 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4968b-e894-417e-ad3a-44cdc29db5f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=276464f90d71465097dae2fe81bdf031)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:21.869746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4968b-e894-417e-ad3a-44cdc29db5f2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:21.884206 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028a7a1-c6c7-45ce-9a7c-786aa6f4479d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:21.900463 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028a7a1-c6c7-45ce-9a7c-786aa6f4479d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:21.910742 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026f0fa-b779-47f6-ac66-eabc4a6e5e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:21.911922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026f0fa-b779-47f6-ac66-eabc4a6e5e1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:21.912275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026f0fa-b779-47f6-ac66-eabc4a6e5e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:21.912612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026f0fa-b779-47f6-ac66-eabc4a6e5e1b admin admin] PATH_INFO: `/v3/projects/b2fc62de7a9140f88a87961dde2d445f/users/d1c15e66eaea440aadca0373a803eed9/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:21.945509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026f0fa-b779-47f6-ac66-eabc4a6e5e1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2fc62de7a9140f88a87961dde2d445f, user_id=d1c15e66eaea440aadca0373a803eed9, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:21.949462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026f0fa-b779-47f6-ac66-eabc4a6e5e1b admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:21.998237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:22.005866 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 818/1654] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:21 2022] PUT /identity/v3/projects/b2fc62de7a9140f88a87961dde2d445f/users/d1c15e66eaea440aadca0373a803eed9/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:22.023826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e9931-df09-45c7-a94a-b853862cfde2 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:22.024196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e9931-df09-45c7-a94a-b853862cfde2 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:22.024482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e9931-df09-45c7-a94a-b853862cfde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:22.067145 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b2f4968b-e894-417e-ad3a-44cdc29db5f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '276464f90d71465097dae2fe81bdf031', 'request_id': 'req-b2f4968b-e894-417e-ad3a-44cdc29db5f2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:22.097097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b2f4968b-e894-417e-ad3a-44cdc29db5f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '276464f90d71465097dae2fe81bdf031', 'request_id': 'req-b2f4968b-e894-417e-ad3a-44cdc29db5f2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:22.137229 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b2f4968b-e894-417e-ad3a-44cdc29db5f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '276464f90d71465097dae2fe81bdf031', 'request_id': 'req-b2f4968b-e894-417e-ad3a-44cdc29db5f2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:22.146187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-857e9931-df09-45c7-a94a-b853862cfde2 None None] MFA Rules not processed for user `d1c15e66eaea440aadca0373a803eed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:22.153740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-857e9931-df09-45c7-a94a-b853862cfde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:22.186408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b2f4968b-e894-417e-ad3a-44cdc29db5f2 admin admin] The token cache is being invalidate because project 276464f90d71465097dae2fe81bdf031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:22.186723 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b2f4968b-e894-417e-ad3a-44cdc29db5f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:22.187682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b2f4968b-e894-417e-ad3a-44cdc29db5f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:22.193171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 837/1655] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:21 2022] DELETE /identity/v3/projects/276464f90d71465097dae2fe81bdf031 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:22.195470 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ff063-9897-4e67-95c1-3f2226c21fd1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:22.195845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ff063-9897-4e67-95c1-3f2226c21fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:22.196932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ff063-9897-4e67-95c1-3f2226c21fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:22.225822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-fb0ff063-9897-4e67-95c1-3f2226c21fd1 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:22.227242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fb0ff063-9897-4e67-95c1-3f2226c21fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:22.319167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 838/1656] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:22.328513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e10ed7-403e-4159-87ab-1d9d1cb06a18 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:22.328910 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e10ed7-403e-4159-87ab-1d9d1cb06a18 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:22.329605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e10ed7-403e-4159-87ab-1d9d1cb06a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:22.361067 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-68e10ed7-403e-4159-87ab-1d9d1cb06a18 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:22.362559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-68e10ed7-403e-4159-87ab-1d9d1cb06a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:22.379082 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 839/1657] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:22.380131 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e10ed7-403e-4159-87ab-1d9d1cb06a18 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:22.384464 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 819/1658] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:22.392870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e10ed7-403e-4159-87ab-1d9d1cb06a18 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:22.393255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e9931-df09-45c7-a94a-b853862cfde2 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:22.395101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-857e9931-df09-45c7-a94a-b853862cfde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:22.410792 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d14290-c7be-4b93-b5bf-96619cc1fe15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:22.415582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14290-c7be-4b93-b5bf-96619cc1fe15 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:22.415761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14290-c7be-4b93-b5bf-96619cc1fe15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:22.416975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14290-c7be-4b93-b5bf-96619cc1fe15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:22.418880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d14290-c7be-4b93-b5bf-96619cc1fe15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:22.420739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d14290-c7be-4b93-b5bf-96619cc1fe15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:22.424980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d14290-c7be-4b93-b5bf-96619cc1fe15 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:22.458065 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 840/1659] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:36:22.463180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d14290-c7be-4b93-b5bf-96619cc1fe15 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:22.493182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d14290-c7be-4b93-b5bf-96619cc1fe15 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:22.499558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90ad08-ca25-4cd8-8ea8-4d13876a0322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:22.501272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ad08-ca25-4cd8-8ea8-4d13876a0322 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:22.501398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ad08-ca25-4cd8-8ea8-4d13876a0322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:22.501602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ad08-ca25-4cd8-8ea8-4d13876a0322 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:22.502160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:22.508755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90ad08-ca25-4cd8-8ea8-4d13876a0322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:22.518011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90ad08-ca25-4cd8-8ea8-4d13876a0322 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:22.581239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e9931-df09-45c7-a94a-b853862cfde2 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:22.590204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda4497-7e75-4d25-a43a-b537b7670b6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:22.590617 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda4497-7e75-4d25-a43a-b537b7670b6d service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:22.590881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda4497-7e75-4d25-a43a-b537b7670b6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:22.591789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda4497-7e75-4d25-a43a-b537b7670b6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:22.599671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 841/1660] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:22 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:22.603581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90ad08-ca25-4cd8-8ea8-4d13876a0322 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:22.626854 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda4497-7e75-4d25-a43a-b537b7670b6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:22.630824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda4497-7e75-4d25-a43a-b537b7670b6d service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:22.657797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 820/1661] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:22.728464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90ad08-ca25-4cd8-8ea8-4d13876a0322 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:22.735986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18417e1-6ac2-4478-a45e-5cfec5fb9640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:22.736914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18417e1-6ac2-4478-a45e-5cfec5fb9640 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:22.737248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18417e1-6ac2-4478-a45e-5cfec5fb9640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:22.737525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18417e1-6ac2-4478-a45e-5cfec5fb9640 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:22.739946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18417e1-6ac2-4478-a45e-5cfec5fb9640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:22.742455 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18417e1-6ac2-4478-a45e-5cfec5fb9640 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:22.788304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 842/1662] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:22 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:22.788811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda4497-7e75-4d25-a43a-b537b7670b6d service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:22.804525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda4497-7e75-4d25-a43a-b537b7670b6d service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:22.824529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac89a2d-b55b-48e6-99c4-456dfcfd8090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:22.826394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac89a2d-b55b-48e6-99c4-456dfcfd8090 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:22.826684 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac89a2d-b55b-48e6-99c4-456dfcfd8090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:22.826937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac89a2d-b55b-48e6-99c4-456dfcfd8090 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:22.827585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac89a2d-b55b-48e6-99c4-456dfcfd8090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:22.830601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac89a2d-b55b-48e6-99c4-456dfcfd8090 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:22.861428 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 821/1663] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:36:22.861630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18417e1-6ac2-4478-a45e-5cfec5fb9640 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:22.878176 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18417e1-6ac2-4478-a45e-5cfec5fb9640 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:22.887357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e7966-aebe-4506-bed6-979b4674b04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:22.888347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e7966-aebe-4506-bed6-979b4674b04e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:22.888347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e7966-aebe-4506-bed6-979b4674b04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:22.888685 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e7966-aebe-4506-bed6-979b4674b04e admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:22.889635 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e7966-aebe-4506-bed6-979b4674b04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:22.890826 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e7966-aebe-4506-bed6-979b4674b04e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:22.896184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:22.918378 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 843/1664] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:36:22.922922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac89a2d-b55b-48e6-99c4-456dfcfd8090 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:22.941136 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac89a2d-b55b-48e6-99c4-456dfcfd8090 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:22.958442 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9325f2-281b-4d60-8af0-b4a1ed925eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:22.959189 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9325f2-281b-4d60-8af0-b4a1ed925eb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:22.959461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9325f2-281b-4d60-8af0-b4a1ed925eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:22.959730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9325f2-281b-4d60-8af0-b4a1ed925eb1 admin admin] PATH_INFO: `/v3/projects/a8a3cb02a86b43fb9e7f0689bafeae64/users/c2a508ae0a1241a89a347348dbca3e7e/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:23.002838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9325f2-281b-4d60-8af0-b4a1ed925eb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a3cb02a86b43fb9e7f0689bafeae64, user_id=c2a508ae0a1241a89a347348dbca3e7e, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:23.006524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9325f2-281b-4d60-8af0-b4a1ed925eb1 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:23.038660 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 822/1665] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:22 2022] PUT /identity/v3/projects/a8a3cb02a86b43fb9e7f0689bafeae64/users/c2a508ae0a1241a89a347348dbca3e7e/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:23.056113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f9d7e-02bf-45bb-8421-ed06dd30057e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:23.056472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f9d7e-02bf-45bb-8421-ed06dd30057e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:23.056753 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f9d7e-02bf-45bb-8421-ed06dd30057e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:23.180067 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5c2f9d7e-02bf-45bb-8421-ed06dd30057e None None] MFA Rules not processed for user `c2a508ae0a1241a89a347348dbca3e7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:23.183851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5c2f9d7e-02bf-45bb-8421-ed06dd30057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:23.377518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 844/1666] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:23.381645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9325f2-281b-4d60-8af0-b4a1ed925eb1 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:23.559466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9325f2-281b-4d60-8af0-b4a1ed925eb1 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:23.565477 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3582533-3f9d-4896-ad70-10226e9fa9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:23.566209 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3582533-3f9d-4896-ad70-10226e9fa9ef admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:23.566580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3582533-3f9d-4896-ad70-10226e9fa9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:23.567201 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3582533-3f9d-4896-ad70-10226e9fa9ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:23.567994 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3582533-3f9d-4896-ad70-10226e9fa9ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:23.570018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3582533-3f9d-4896-ad70-10226e9fa9ef admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:23.640930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 823/1667] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:23 2022] POST /identity/v3/projects => generated 377 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:23.645673 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f9d7e-02bf-45bb-8421-ed06dd30057e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:23.669499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f9d7e-02bf-45bb-8421-ed06dd30057e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:23.745268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc1e8-2d6d-448a-8558-7125160ff9c6 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:23.745584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc1e8-2d6d-448a-8558-7125160ff9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:23.745860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc1e8-2d6d-448a-8558-7125160ff9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:23.787335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e7d98-fbac-49f5-8283-830275622128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:23.790785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e7d98-fbac-49f5-8283-830275622128 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:23.791512 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e7d98-fbac-49f5-8283-830275622128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:23.792527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e7d98-fbac-49f5-8283-830275622128 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:23.794665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e7d98-fbac-49f5-8283-830275622128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:23.798767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e7d98-fbac-49f5-8283-830275622128 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:23.828177 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-218fc1e8-2d6d-448a-8558-7125160ff9c6 None None] MFA Rules not processed for user `52760b6736e14ccd8df8aca0015cb292`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:23.829874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-218fc1e8-2d6d-448a-8558-7125160ff9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:23.841777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 845/1668] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:23 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:23.844788 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e7d98-fbac-49f5-8283-830275622128 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:23.865530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e7d98-fbac-49f5-8283-830275622128 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:23.892518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00a661-4c06-4337-a136-aa832080d84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:23.893172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a661-4c06-4337-a136-aa832080d84e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:23.897560 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a661-4c06-4337-a136-aa832080d84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:23.897932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a661-4c06-4337-a136-aa832080d84e admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:23.898654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00a661-4c06-4337-a136-aa832080d84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:23.903419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00a661-4c06-4337-a136-aa832080d84e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:23.945012 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 846/1669] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:23.945934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00a661-4c06-4337-a136-aa832080d84e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:23.964645 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00a661-4c06-4337-a136-aa832080d84e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:23.968498 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 824/1670] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:23.982924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb0027-9285-4fd9-8357-742cb28cf0de None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:23.983133 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb0027-9285-4fd9-8357-742cb28cf0de None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:23.983370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb0027-9285-4fd9-8357-742cb28cf0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:23.991033 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157756f-f539-4d08-ba13-480fbe19e8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:23.992726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157756f-f539-4d08-ba13-480fbe19e8ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:23.995144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157756f-f539-4d08-ba13-480fbe19e8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:23.995459 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157756f-f539-4d08-ba13-480fbe19e8ac admin admin] PATH_INFO: `/v3/projects/96c08444b98b4a21b4767b1fe84cd9bb/users/e1a6aa0986904ee2b6c288d03e191350/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:24.054292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-76eb0027-9285-4fd9-8357-742cb28cf0de None None] MFA Rules not processed for user `52760b6736e14ccd8df8aca0015cb292`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:24.064155 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-76eb0027-9285-4fd9-8357-742cb28cf0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:24.064387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157756f-f539-4d08-ba13-480fbe19e8ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c08444b98b4a21b4767b1fe84cd9bb, user_id=e1a6aa0986904ee2b6c288d03e191350, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:24.064989 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157756f-f539-4d08-ba13-480fbe19e8ac admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:24.070324 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 825/1671] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:24.082932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb0027-9285-4fd9-8357-742cb28cf0de None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:24.085117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-76eb0027-9285-4fd9-8357-742cb28cf0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:24.096264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 847/1672] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:23 2022] PUT /identity/v3/projects/96c08444b98b4a21b4767b1fe84cd9bb/users/e1a6aa0986904ee2b6c288d03e191350/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:24.100217 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157756f-f539-4d08-ba13-480fbe19e8ac admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:24.103579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:24.233224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:24.284201 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157756f-f539-4d08-ba13-480fbe19e8ac admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:24.287091 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb0027-9285-4fd9-8357-742cb28cf0de None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:24.289694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c0086-5c1e-4120-81f1-e294a3a814b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:24.290391 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c0086-5c1e-4120-81f1-e294a3a814b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:24.290734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c0086-5c1e-4120-81f1-e294a3a814b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:24.291073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c0086-5c1e-4120-81f1-e294a3a814b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:24.292003 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c0086-5c1e-4120-81f1-e294a3a814b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:24.294284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c0086-5c1e-4120-81f1-e294a3a814b1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:24.324639 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 848/1673] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:36:24.326606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c0086-5c1e-4120-81f1-e294a3a814b1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:24.342525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c0086-5c1e-4120-81f1-e294a3a814b1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:24.356596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c11e3-b3a3-4f46-b2e7-16fa53a28aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:24.357536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c11e3-b3a3-4f46-b2e7-16fa53a28aa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:24.357904 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c11e3-b3a3-4f46-b2e7-16fa53a28aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:24.358244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c11e3-b3a3-4f46-b2e7-16fa53a28aa6 admin admin] PATH_INFO: `/v3/projects/96c08444b98b4a21b4767b1fe84cd9bb/users/e1a6aa0986904ee2b6c288d03e191350/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:24.361676 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c11e3-b3a3-4f46-b2e7-16fa53a28aa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c08444b98b4a21b4767b1fe84cd9bb, user_id=e1a6aa0986904ee2b6c288d03e191350, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:24.364458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c11e3-b3a3-4f46-b2e7-16fa53a28aa6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:24.383981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 849/1674] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:24 2022] PUT /identity/v3/projects/96c08444b98b4a21b4767b1fe84cd9bb/users/e1a6aa0986904ee2b6c288d03e191350/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:24.397864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169ed66-d6ea-421b-8c19-9d4fc95fa0cc None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:24.398678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169ed66-d6ea-421b-8c19-9d4fc95fa0cc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:24.399751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169ed66-d6ea-421b-8c19-9d4fc95fa0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:24.404379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0838b7a-ab7c-420a-bf74-a7ca75e926b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:24.405080 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0838b7a-ab7c-420a-bf74-a7ca75e926b8 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:24.405435 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0838b7a-ab7c-420a-bf74-a7ca75e926b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:24.406421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0838b7a-ab7c-420a-bf74-a7ca75e926b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:24.450609 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f169ed66-d6ea-421b-8c19-9d4fc95fa0cc None None] MFA Rules not processed for user `e1a6aa0986904ee2b6c288d03e191350`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:24.453199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f169ed66-d6ea-421b-8c19-9d4fc95fa0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:24.524222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0838b7a-ab7c-420a-bf74-a7ca75e926b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:24.535938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0838b7a-ab7c-420a-bf74-a7ca75e926b8 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:24.557873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 826/1675] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:36:24.571944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 850/1676] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:24.592262 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0838b7a-ab7c-420a-bf74-a7ca75e926b8 service nova] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:24.658115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:24.682790 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0838b7a-ab7c-420a-bf74-a7ca75e926b8 service nova] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:24.790511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6faf1-8668-40fc-b614-097b0999d7de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:24.791181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6faf1-8668-40fc-b614-097b0999d7de service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:24.791501 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6faf1-8668-40fc-b614-097b0999d7de service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:24.791763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6faf1-8668-40fc-b614-097b0999d7de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:24.949321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6faf1-8668-40fc-b614-097b0999d7de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:24.952790 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6faf1-8668-40fc-b614-097b0999d7de service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:24.984921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 827/1677] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:26.142578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94040f2a-a16a-4323-bc09-497cac7f7ba2 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:26.143177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94040f2a-a16a-4323-bc09-497cac7f7ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:26.143475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94040f2a-a16a-4323-bc09-497cac7f7ba2 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:26.145549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 851/1678] 10.222.0.62 () {54 vars in 912 bytes} [Thu Sep 1 16:36:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:36:26.173604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016a9ce-9c6b-4834-8b76-baa79346cd74 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:26.174054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016a9ce-9c6b-4834-8b76-baa79346cd74 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:26.174131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016a9ce-9c6b-4834-8b76-baa79346cd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:26.268119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-b016a9ce-9c6b-4834-8b76-baa79346cd74 None None] MFA Rules not processed for user `0718a95189434a5ea901c40bc868bea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:26.271656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b016a9ce-9c6b-4834-8b76-baa79346cd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:26.358034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 828/1679] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:36:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:26.375237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94040f2a-a16a-4323-bc09-497cac7f7ba2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:26.385138 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-94040f2a-a16a-4323-bc09-497cac7f7ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:26.433280 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94040f2a-a16a-4323-bc09-497cac7f7ba2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:26.442357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360f9ec-0855-4853-a658-9a12c6a75a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:26.443101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360f9ec-0855-4853-a658-9a12c6a75a46 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:26.443418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360f9ec-0855-4853-a658-9a12c6a75a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:26.445241 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360f9ec-0855-4853-a658-9a12c6a75a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:26.467670 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360f9ec-0855-4853-a658-9a12c6a75a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:26.469313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360f9ec-0855-4853-a658-9a12c6a75a46 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:26.480082 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 852/1680] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:26.646691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576cb7c-49a5-4144-bdd0-e9ab12ca05d5 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:26.647418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576cb7c-49a5-4144-bdd0-e9ab12ca05d5 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:26.647418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576cb7c-49a5-4144-bdd0-e9ab12ca05d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:26.711699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-1576cb7c-49a5-4144-bdd0-e9ab12ca05d5 None None] MFA Rules not processed for user `2c1c3f8f28b147fc838533baf62b72af`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:26.714820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1576cb7c-49a5-4144-bdd0-e9ab12ca05d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:26.720028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26480ed-31f4-4d36-adc6-1bedcb79b5c8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:26.720439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26480ed-31f4-4d36-adc6-1bedcb79b5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:26.720794 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26480ed-31f4-4d36-adc6-1bedcb79b5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:26.799914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a26480ed-31f4-4d36-adc6-1bedcb79b5c8 None None] MFA Rules not processed for user `3a9b222396b1438f9fa0af0e197a9293`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:26.801322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a26480ed-31f4-4d36-adc6-1bedcb79b5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:26.854274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:26.930683 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 829/1681] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:26.946196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576cb7c-49a5-4144-bdd0-e9ab12ca05d5 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:26.947727 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1576cb7c-49a5-4144-bdd0-e9ab12ca05d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:27.024347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 853/1682] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:27.233668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576cb7c-49a5-4144-bdd0-e9ab12ca05d5 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:27.243368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a94576-32f0-4b14-a6c0-16540f736241 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:27.244982 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a94576-32f0-4b14-a6c0-16540f736241 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:27.245495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a94576-32f0-4b14-a6c0-16540f736241 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:27.246208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a94576-32f0-4b14-a6c0-16540f736241 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:27.274367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a94576-32f0-4b14-a6c0-16540f736241 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:27.278362 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a94576-32f0-4b14-a6c0-16540f736241 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:27.304812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 830/1683] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:27.437379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26480ed-31f4-4d36-adc6-1bedcb79b5c8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:27.475447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26480ed-31f4-4d36-adc6-1bedcb79b5c8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:27.498156 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b9c0f-f9e6-470d-8e7f-5ff8d1ea089e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:27.499038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b9c0f-f9e6-470d-8e7f-5ff8d1ea089e service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:27.499324 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b9c0f-f9e6-470d-8e7f-5ff8d1ea089e service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:27.499595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b9c0f-f9e6-470d-8e7f-5ff8d1ea089e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:27.532781 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b9c0f-f9e6-470d-8e7f-5ff8d1ea089e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:27.534752 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b9c0f-f9e6-470d-8e7f-5ff8d1ea089e service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:27.571043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 854/1684] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:36:28.472553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f74c36-f186-43b6-8d3e-8d5aff546e97 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:28.473505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f74c36-f186-43b6-8d3e-8d5aff546e97 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:28.474112 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f74c36-f186-43b6-8d3e-8d5aff546e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:28.530855 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-46f74c36-f186-43b6-8d3e-8d5aff546e97 None None] MFA Rules not processed for user `849f875ede78424c81919df6d9c152dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:28.537904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-46f74c36-f186-43b6-8d3e-8d5aff546e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:28.724793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 831/1685] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:28.965722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca0eb1-22ec-4bba-8e32-1a20b15a1b80 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:28.966405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca0eb1-22ec-4bba-8e32-1a20b15a1b80 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:28.966580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca0eb1-22ec-4bba-8e32-1a20b15a1b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:29.055155 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b2ca0eb1-22ec-4bba-8e32-1a20b15a1b80 None None] MFA Rules not processed for user `f21b1fea950d48ef9a55cded8ed172a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:29.063167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:29.066822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b2ca0eb1-22ec-4bba-8e32-1a20b15a1b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:29.196921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f74c36-f186-43b6-8d3e-8d5aff546e97 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:29.198377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-46f74c36-f186-43b6-8d3e-8d5aff546e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:29.297499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 855/1686] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:29.299387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0780c0-339c-472e-881c-32c6eceb2376 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:29.299819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0780c0-339c-472e-881c-32c6eceb2376 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:29.300140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0780c0-339c-472e-881c-32c6eceb2376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:29.358467 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dd0780c0-339c-472e-881c-32c6eceb2376 None None] MFA Rules not processed for user `f21b1fea950d48ef9a55cded8ed172a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:29.360159 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dd0780c0-339c-472e-881c-32c6eceb2376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:29.394808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 856/1687] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:29.412338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f74c36-f186-43b6-8d3e-8d5aff546e97 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:29.418280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04be4f8-5da7-4361-a77c-fe04b23b2ba5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6281dcf2927a4ed2870125df8801be78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:29.419340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04be4f8-5da7-4361-a77c-fe04b23b2ba5 service placement] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:29.419623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04be4f8-5da7-4361-a77c-fe04b23b2ba5 service placement] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:29.419917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04be4f8-5da7-4361-a77c-fe04b23b2ba5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:29.422488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a04be4f8-5da7-4361-a77c-fe04b23b2ba5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:29.443743 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04be4f8-5da7-4361-a77c-fe04b23b2ba5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:29.447678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04be4f8-5da7-4361-a77c-fe04b23b2ba5 service placement] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:29.465879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:29.474712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 832/1688] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:29.566953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0780c0-339c-472e-881c-32c6eceb2376 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:29.581294 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5756160-2027-416a-9716-15df467d6237 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:29.581977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5756160-2027-416a-9716-15df467d6237 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:29.582552 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5756160-2027-416a-9716-15df467d6237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:29.594558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0780c0-339c-472e-881c-32c6eceb2376 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:29.609773 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6a89b-92f5-4e9c-85f8-847bb2f60c16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:29.611732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6a89b-92f5-4e9c-85f8-847bb2f60c16 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:29.612119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6a89b-92f5-4e9c-85f8-847bb2f60c16 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:29.612904 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6a89b-92f5-4e9c-85f8-847bb2f60c16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:29.625200 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-a5756160-2027-416a-9716-15df467d6237 None None] MFA Rules not processed for user `52760b6736e14ccd8df8aca0015cb292`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:29.626931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a5756160-2027-416a-9716-15df467d6237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:29.642764 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6a89b-92f5-4e9c-85f8-847bb2f60c16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:29.647414 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6a89b-92f5-4e9c-85f8-847bb2f60c16 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:29.683322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 857/1689] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:29.785528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 833/1690] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:30.444715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6a89b-92f5-4e9c-85f8-847bb2f60c16 service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:30.463740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6a89b-92f5-4e9c-85f8-847bb2f60c16 service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:30.468914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe824e-ef54-4927-a038-d604a487656a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:30.469580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe824e-ef54-4927-a038-d604a487656a service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:30.469865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe824e-ef54-4927-a038-d604a487656a service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:30.470137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe824e-ef54-4927-a038-d604a487656a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:30.494226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe824e-ef54-4927-a038-d604a487656a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:30.495752 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe824e-ef54-4927-a038-d604a487656a service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:30.523906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 858/1691] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:30.541470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62468319-8ad6-4bd6-b7ff-d07836deaff4 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:30.542412 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62468319-8ad6-4bd6-b7ff-d07836deaff4 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:30.542703 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62468319-8ad6-4bd6-b7ff-d07836deaff4 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:30.544914 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 834/1692] 10.222.0.62 () {54 vars in 912 bytes} [Thu Sep 1 16:36:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:36:30.571678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3a64e-355e-44ed-b5c9-36bfcc7cfb1c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:30.571998 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3a64e-355e-44ed-b5c9-36bfcc7cfb1c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:30.572258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3a64e-355e-44ed-b5c9-36bfcc7cfb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:30.629697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-85a3a64e-355e-44ed-b5c9-36bfcc7cfb1c None None] MFA Rules not processed for user `0718a95189434a5ea901c40bc868bea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:30.632134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-85a3a64e-355e-44ed-b5c9-36bfcc7cfb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:30.658403 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 859/1693] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:36:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:30.719955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62468319-8ad6-4bd6-b7ff-d07836deaff4 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:30.738395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62468319-8ad6-4bd6-b7ff-d07836deaff4 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:30.748613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42609a64-1ba0-433b-8962-918f6fde7180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:30.749272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42609a64-1ba0-433b-8962-918f6fde7180 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:30.749552 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42609a64-1ba0-433b-8962-918f6fde7180 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:30.749823 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42609a64-1ba0-433b-8962-918f6fde7180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:30.776838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42609a64-1ba0-433b-8962-918f6fde7180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:30.778906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42609a64-1ba0-433b-8962-918f6fde7180 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:30.811864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 835/1694] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:30.944053 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94ac49-52df-47bd-b07b-b0c4a1d3b316 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:30.944464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94ac49-52df-47bd-b07b-b0c4a1d3b316 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:30.944780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94ac49-52df-47bd-b07b-b0c4a1d3b316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:30.984422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6e94ac49-52df-47bd-b07b-b0c4a1d3b316 None None] MFA Rules not processed for user `2c1c3f8f28b147fc838533baf62b72af`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:30.985763 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6e94ac49-52df-47bd-b07b-b0c4a1d3b316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:31.005115 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 860/1695] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:32.364527 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93ac43-88be-4a02-8ae7-ce98cea2c808 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:32.364527 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93ac43-88be-4a02-8ae7-ce98cea2c808 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:32.364527 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93ac43-88be-4a02-8ae7-ce98cea2c808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:32.426593 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-ec93ac43-88be-4a02-8ae7-ce98cea2c808 None None] MFA Rules not processed for user `3a9b222396b1438f9fa0af0e197a9293`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:32.428173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ec93ac43-88be-4a02-8ae7-ce98cea2c808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:32.454356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 836/1696] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:35.624284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94ac49-52df-47bd-b07b-b0c4a1d3b316 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:35.627964 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6e94ac49-52df-47bd-b07b-b0c4a1d3b316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:35.901042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94ac49-52df-47bd-b07b-b0c4a1d3b316 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:35.907647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fc62f-ded6-4f4f-a33f-15ee745c85b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:35.908476 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fc62f-ded6-4f4f-a33f-15ee745c85b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:35.908753 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fc62f-ded6-4f4f-a33f-15ee745c85b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:35.909084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fc62f-ded6-4f4f-a33f-15ee745c85b1 admin admin] PATH_INFO: `/v3/users/7b20bf54d9a94e62a119026861dacd2a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:35.910621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fc62f-ded6-4f4f-a33f-15ee745c85b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b20bf54d9a94e62a119026861dacd2a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:35.912886 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fc62f-ded6-4f4f-a33f-15ee745c85b1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:36.042863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d17fc62f-ded6-4f4f-a33f-15ee745c85b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b20bf54d9a94e62a119026861dacd2a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:36.105337 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d17fc62f-ded6-4f4f-a33f-15ee745c85b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b20bf54d9a94e62a119026861dacd2a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:36.109093 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d17fc62f-ded6-4f4f-a33f-15ee745c85b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b20bf54d9a94e62a119026861dacd2a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:36.141720 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:36.192761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d17fc62f-ded6-4f4f-a33f-15ee745c85b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b20bf54d9a94e62a119026861dacd2a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:36.219670 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d17fc62f-ded6-4f4f-a33f-15ee745c85b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b20bf54d9a94e62a119026861dacd2a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:36.260248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 861/1697] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:35 2022] DELETE /identity/v3/users/7b20bf54d9a94e62a119026861dacd2a => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:36.265699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74799eea-b5b4-4357-b929-92b97b435da5 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:36.266237 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74799eea-b5b4-4357-b929-92b97b435da5 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:36.266339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74799eea-b5b4-4357-b929-92b97b435da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:36.321242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-74799eea-b5b4-4357-b929-92b97b435da5 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:36.328125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-74799eea-b5b4-4357-b929-92b97b435da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:36.551721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 837/1698] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:36.565374 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fc62f-ded6-4f4f-a33f-15ee745c85b1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:36.568576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d17fc62f-ded6-4f4f-a33f-15ee745c85b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:36.876702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fc62f-ded6-4f4f-a33f-15ee745c85b1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:36.890164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d71e55-7e5e-4c49-898b-f8d2faf7ae96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:36.891371 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d71e55-7e5e-4c49-898b-f8d2faf7ae96 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:36.892498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d71e55-7e5e-4c49-898b-f8d2faf7ae96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:36.894040 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d71e55-7e5e-4c49-898b-f8d2faf7ae96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:36.944489 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d71e55-7e5e-4c49-898b-f8d2faf7ae96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:36.950501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d71e55-7e5e-4c49-898b-f8d2faf7ae96 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:36.999276 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 862/1699] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:38.273719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74799eea-b5b4-4357-b929-92b97b435da5 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:38.275995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-74799eea-b5b4-4357-b929-92b97b435da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:38.297149 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74799eea-b5b4-4357-b929-92b97b435da5 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:38.318839 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:38.319521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:38.319784 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:38.320669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] PATH_INFO: `/v3/projects/2ac77723842947a5b70a97ad0701711b` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:38.324605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac77723842947a5b70a97ad0701711b)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:38.327464 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:38.394992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac77723842947a5b70a97ad0701711b', 'request_id': 'req-68663da7-9573-4b82-b8e2-baa2cf9d32ce'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:38.415096 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac77723842947a5b70a97ad0701711b', 'request_id': 'req-68663da7-9573-4b82-b8e2-baa2cf9d32ce'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:38.424171 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac77723842947a5b70a97ad0701711b', 'request_id': 'req-68663da7-9573-4b82-b8e2-baa2cf9d32ce'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:38.442969 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] The token cache is being invalidate because project 2ac77723842947a5b70a97ad0701711b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:38.443754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:38.445030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:38.447471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 838/1700] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:38 2022] DELETE /identity/v3/projects/2ac77723842947a5b70a97ad0701711b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:38.452242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d71e55-7e5e-4c49-898b-f8d2faf7ae96 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:38.465418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d71e55-7e5e-4c49-898b-f8d2faf7ae96 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:38.545028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be190775-ccf5-48dc-9cd1-af0ad08148c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:38.545885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be190775-ccf5-48dc-9cd1-af0ad08148c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:38.546291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be190775-ccf5-48dc-9cd1-af0ad08148c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:38.546627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be190775-ccf5-48dc-9cd1-af0ad08148c4 admin admin] PATH_INFO: `/v3/users/05db1b00305f4d088ffa6efd69774e43` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:38.548281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be190775-ccf5-48dc-9cd1-af0ad08148c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05db1b00305f4d088ffa6efd69774e43)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:38.550255 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be190775-ccf5-48dc-9cd1-af0ad08148c4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:38.615007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-be190775-ccf5-48dc-9cd1-af0ad08148c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05db1b00305f4d088ffa6efd69774e43'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:38.626747 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-be190775-ccf5-48dc-9cd1-af0ad08148c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05db1b00305f4d088ffa6efd69774e43'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:38.627435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-be190775-ccf5-48dc-9cd1-af0ad08148c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05db1b00305f4d088ffa6efd69774e43'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:38.686908 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-be190775-ccf5-48dc-9cd1-af0ad08148c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05db1b00305f4d088ffa6efd69774e43'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:38.695303 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-be190775-ccf5-48dc-9cd1-af0ad08148c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05db1b00305f4d088ffa6efd69774e43'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:38.697510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:38.732103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 863/1701] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:38 2022] DELETE /identity/v3/users/05db1b00305f4d088ffa6efd69774e43 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:39.093467 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:39.094936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:39.130915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68663da7-9573-4b82-b8e2-baa2cf9d32ce admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:39.138147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:39.209078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:39.209877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:39.210073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:39.210332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed admin admin] PATH_INFO: `/v3/projects/62e10b2106c64dcdbeb4bc50294515ec` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:39.211508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=62e10b2106c64dcdbeb4bc50294515ec)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:39.213577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:39.272919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62e10b2106c64dcdbeb4bc50294515ec', 'request_id': 'req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:39.282337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62e10b2106c64dcdbeb4bc50294515ec', 'request_id': 'req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:39.296198 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62e10b2106c64dcdbeb4bc50294515ec', 'request_id': 'req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:39.307903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed admin admin] The token cache is being invalidate because project 62e10b2106c64dcdbeb4bc50294515ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:39.308285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:39.309021 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-16c640ee-5fbe-4c66-a1c6-e573d800b0ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:39.310550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 839/1702] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:39 2022] DELETE /identity/v3/projects/62e10b2106c64dcdbeb4bc50294515ec => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:39.315880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bc00f-52c4-4811-812f-a63acc012f62 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:39.316067 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bc00f-52c4-4811-812f-a63acc012f62 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:39.316340 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bc00f-52c4-4811-812f-a63acc012f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:39.346104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-793bc00f-52c4-4811-812f-a63acc012f62 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:39.348421 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-793bc00f-52c4-4811-812f-a63acc012f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:39.422534 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 864/1703] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:39.430157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dee33b-86f9-434d-80bb-c18a4a1c3113 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:39.430441 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dee33b-86f9-434d-80bb-c18a4a1c3113 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:39.430704 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dee33b-86f9-434d-80bb-c18a4a1c3113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:39.451334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-33dee33b-86f9-434d-80bb-c18a4a1c3113 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:39.452597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-33dee33b-86f9-434d-80bb-c18a4a1c3113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:39.462040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 840/1704] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:39.466989 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bc00f-52c4-4811-812f-a63acc012f62 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:39.476977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bc00f-52c4-4811-812f-a63acc012f62 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:39.482179 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a79249-92d2-4f52-b3bc-f5bd934a9524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:39.482800 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a79249-92d2-4f52-b3bc-f5bd934a9524 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:39.483071 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a79249-92d2-4f52-b3bc-f5bd934a9524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:39.483339 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a79249-92d2-4f52-b3bc-f5bd934a9524 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:39.484070 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a79249-92d2-4f52-b3bc-f5bd934a9524 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:39.484390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a79249-92d2-4f52-b3bc-f5bd934a9524 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:39.485994 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a79249-92d2-4f52-b3bc-f5bd934a9524 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:39.498155 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 865/1705] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:36:39.502398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dee33b-86f9-434d-80bb-c18a4a1c3113 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:39.512671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dee33b-86f9-434d-80bb-c18a4a1c3113 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:39.517586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b4caa-673d-43f2-9b25-7ffbc34f9cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:39.518199 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b4caa-673d-43f2-9b25-7ffbc34f9cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:39.518486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b4caa-673d-43f2-9b25-7ffbc34f9cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:39.518758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b4caa-673d-43f2-9b25-7ffbc34f9cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:39.519391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b4caa-673d-43f2-9b25-7ffbc34f9cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:39.521404 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b4caa-673d-43f2-9b25-7ffbc34f9cc2 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:39.556983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 841/1706] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:39 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:39.558216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a79249-92d2-4f52-b3bc-f5bd934a9524 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:39.570758 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a79249-92d2-4f52-b3bc-f5bd934a9524 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:39.639341 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da6c0e-27d8-4d2b-870a-5c5fa7bfeed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:39.640063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da6c0e-27d8-4d2b-870a-5c5fa7bfeed6 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:39.640405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da6c0e-27d8-4d2b-870a-5c5fa7bfeed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:39.640732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da6c0e-27d8-4d2b-870a-5c5fa7bfeed6 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:39.641437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da6c0e-27d8-4d2b-870a-5c5fa7bfeed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:39.643406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da6c0e-27d8-4d2b-870a-5c5fa7bfeed6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:39.670821 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 866/1707] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:39 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:39.672467 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b4caa-673d-43f2-9b25-7ffbc34f9cc2 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:39.682962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b4caa-673d-43f2-9b25-7ffbc34f9cc2 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:39.687988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7996d9d-44fd-4b85-98d3-8e075b61940f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:39.688784 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7996d9d-44fd-4b85-98d3-8e075b61940f admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:39.689064 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7996d9d-44fd-4b85-98d3-8e075b61940f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:39.689342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7996d9d-44fd-4b85-98d3-8e075b61940f admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:39.690010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7996d9d-44fd-4b85-98d3-8e075b61940f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:39.691653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7996d9d-44fd-4b85-98d3-8e075b61940f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:39.706451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 842/1708] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:39.708774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da6c0e-27d8-4d2b-870a-5c5fa7bfeed6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:39.719678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da6c0e-27d8-4d2b-870a-5c5fa7bfeed6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:39.724565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1093fb5-e7c2-4f24-831c-6e7e1936940e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:39.725538 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1093fb5-e7c2-4f24-831c-6e7e1936940e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:39.725837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1093fb5-e7c2-4f24-831c-6e7e1936940e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:39.726103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1093fb5-e7c2-4f24-831c-6e7e1936940e admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:39.726744 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1093fb5-e7c2-4f24-831c-6e7e1936940e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:39.728402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1093fb5-e7c2-4f24-831c-6e7e1936940e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:39.743795 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 867/1709] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:39.744950 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7996d9d-44fd-4b85-98d3-8e075b61940f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:39.764716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7996d9d-44fd-4b85-98d3-8e075b61940f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:39.774094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6b3f4-70d7-4190-a42b-b956d84d6410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:39.775517 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6b3f4-70d7-4190-a42b-b956d84d6410 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:39.776166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6b3f4-70d7-4190-a42b-b956d84d6410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:39.776750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6b3f4-70d7-4190-a42b-b956d84d6410 admin admin] PATH_INFO: `/v3/projects/c194e18f6f6f400594ea75a61fa58427/users/b8efd053afdf4550b0b65f043de6e516/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:39.802176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6b3f4-70d7-4190-a42b-b956d84d6410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c194e18f6f6f400594ea75a61fa58427, user_id=b8efd053afdf4550b0b65f043de6e516, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:39.806189 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6b3f4-70d7-4190-a42b-b956d84d6410 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:39.833166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 843/1710] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:39 2022] PUT /identity/v3/projects/c194e18f6f6f400594ea75a61fa58427/users/b8efd053afdf4550b0b65f043de6e516/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:39.844560 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797fdf0-74a7-4f45-820d-e1071f6b18c2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:39.845018 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797fdf0-74a7-4f45-820d-e1071f6b18c2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:39.845297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797fdf0-74a7-4f45-820d-e1071f6b18c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:39.898498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4797fdf0-74a7-4f45-820d-e1071f6b18c2 None None] MFA Rules not processed for user `b8efd053afdf4550b0b65f043de6e516`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:39.900445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4797fdf0-74a7-4f45-820d-e1071f6b18c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:39.981186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:40.000358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 868/1711] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:40.006267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6b3f4-70d7-4190-a42b-b956d84d6410 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:40.052854 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:40.091337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6b3f4-70d7-4190-a42b-b956d84d6410 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:40.096443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4860b6-1b50-4cc3-b535-50bf4db26522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:40.097098 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4860b6-1b50-4cc3-b535-50bf4db26522 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:40.098601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4860b6-1b50-4cc3-b535-50bf4db26522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:40.098601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4860b6-1b50-4cc3-b535-50bf4db26522 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:40.098601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4860b6-1b50-4cc3-b535-50bf4db26522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:40.100892 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4860b6-1b50-4cc3-b535-50bf4db26522 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:40.191613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 844/1712] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:40 2022] POST /identity/v3/projects => generated 369 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:36:40.192009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797fdf0-74a7-4f45-820d-e1071f6b18c2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:40.229914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797fdf0-74a7-4f45-820d-e1071f6b18c2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:40.348148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e99e8-8a16-42c7-affd-4efcfa113944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:40.351598 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e99e8-8a16-42c7-affd-4efcfa113944 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:40.352044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e99e8-8a16-42c7-affd-4efcfa113944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:40.352459 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e99e8-8a16-42c7-affd-4efcfa113944 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:40.353226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e99e8-8a16-42c7-affd-4efcfa113944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:40.355742 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e99e8-8a16-42c7-affd-4efcfa113944 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:40.422765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 869/1713] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:40 2022] POST /identity/v3/users => generated 291 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:36:40.427044 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4860b6-1b50-4cc3-b535-50bf4db26522 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:40.444580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4860b6-1b50-4cc3-b535-50bf4db26522 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:40.462680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fc9a0-0ec9-49af-8332-b6ec7d3a4037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:40.463735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fc9a0-0ec9-49af-8332-b6ec7d3a4037 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:40.464336 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fc9a0-0ec9-49af-8332-b6ec7d3a4037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:40.466641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fc9a0-0ec9-49af-8332-b6ec7d3a4037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:40.469103 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5fc9a0-0ec9-49af-8332-b6ec7d3a4037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:40.472799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5fc9a0-0ec9-49af-8332-b6ec7d3a4037 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:40.495496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53772bcb-de29-4852-972c-64b138a71e76 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:40.496455 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53772bcb-de29-4852-972c-64b138a71e76 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:40.497338 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53772bcb-de29-4852-972c-64b138a71e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:40.504302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 845/1714] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:40.509196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fc9a0-0ec9-49af-8332-b6ec7d3a4037 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:40.531209 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fc9a0-0ec9-49af-8332-b6ec7d3a4037 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:40.545371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ee4ab-b4d9-4076-a663-0355a4fb2c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:40.546062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ee4ab-b4d9-4076-a663-0355a4fb2c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:40.546529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ee4ab-b4d9-4076-a663-0355a4fb2c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:40.547077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ee4ab-b4d9-4076-a663-0355a4fb2c4a admin admin] PATH_INFO: `/v3/projects/8677bc0d96ed461d96cd50e46b4807bf/users/397219fe731d40ddbbac2170ff10f38a/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:40.580081 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ee4ab-b4d9-4076-a663-0355a4fb2c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8677bc0d96ed461d96cd50e46b4807bf, user_id=397219fe731d40ddbbac2170ff10f38a, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:40.588136 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ee4ab-b4d9-4076-a663-0355a4fb2c4a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:40.622040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 846/1715] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:40 2022] PUT /identity/v3/projects/8677bc0d96ed461d96cd50e46b4807bf/users/397219fe731d40ddbbac2170ff10f38a/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:40.623728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-53772bcb-de29-4852-972c-64b138a71e76 None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:40.626172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ee4ab-b4d9-4076-a663-0355a4fb2c4a admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:40.626318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-53772bcb-de29-4852-972c-64b138a71e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:40.808485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ee4ab-b4d9-4076-a663-0355a4fb2c4a admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:40.824224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab28eb4-fd5a-4c3d-b4f9-89b0ef00152f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:40.824923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28eb4-fd5a-4c3d-b4f9-89b0ef00152f admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:40.825259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28eb4-fd5a-4c3d-b4f9-89b0ef00152f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:40.825542 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab28eb4-fd5a-4c3d-b4f9-89b0ef00152f admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:40.826210 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab28eb4-fd5a-4c3d-b4f9-89b0ef00152f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:40.842633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab28eb4-fd5a-4c3d-b4f9-89b0ef00152f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:40.872930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 847/1716] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:40.880519 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab28eb4-fd5a-4c3d-b4f9-89b0ef00152f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:40.911475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab28eb4-fd5a-4c3d-b4f9-89b0ef00152f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:40.928645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e9c69-c5af-4e39-b036-cbf434a0c971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:40.929413 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e9c69-c5af-4e39-b036-cbf434a0c971 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:40.929676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e9c69-c5af-4e39-b036-cbf434a0c971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:40.929942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e9c69-c5af-4e39-b036-cbf434a0c971 admin admin] PATH_INFO: `/v3/projects/8677bc0d96ed461d96cd50e46b4807bf/users/397219fe731d40ddbbac2170ff10f38a/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:40.933673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e9c69-c5af-4e39-b036-cbf434a0c971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8677bc0d96ed461d96cd50e46b4807bf, user_id=397219fe731d40ddbbac2170ff10f38a, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:40.943719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e9c69-c5af-4e39-b036-cbf434a0c971 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:40.951281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 870/1717] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:36:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:40.975810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 848/1718] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:40 2022] PUT /identity/v3/projects/8677bc0d96ed461d96cd50e46b4807bf/users/397219fe731d40ddbbac2170ff10f38a/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:40.996764 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b8bf3-5c14-4fc3-aa6a-d5b6e63de16b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:41.000526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b8bf3-5c14-4fc3-aa6a-d5b6e63de16b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:41.000863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b8bf3-5c14-4fc3-aa6a-d5b6e63de16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:41.118772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cb3b8bf3-5c14-4fc3-aa6a-d5b6e63de16b None None] MFA Rules not processed for user `397219fe731d40ddbbac2170ff10f38a`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:41.125864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cb3b8bf3-5c14-4fc3-aa6a-d5b6e63de16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:41.238745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:41.267477 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e9c69-c5af-4e39-b036-cbf434a0c971 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:41.271719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-973e9c69-c5af-4e39-b036-cbf434a0c971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:41.315554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 871/1719] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:41.338830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b8bf3-5c14-4fc3-aa6a-d5b6e63de16b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:41.343376 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cb3b8bf3-5c14-4fc3-aa6a-d5b6e63de16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:41.508006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b8bf3-5c14-4fc3-aa6a-d5b6e63de16b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:41.530004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e0546-ee37-4c2a-bfb1-0e2dee8187c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:41.533250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e0546-ee37-4c2a-bfb1-0e2dee8187c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:41.533542 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e0546-ee37-4c2a-bfb1-0e2dee8187c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:41.533814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e0546-ee37-4c2a-bfb1-0e2dee8187c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:41.550793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:41.565732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e0546-ee37-4c2a-bfb1-0e2dee8187c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:41.569479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e0546-ee37-4c2a-bfb1-0e2dee8187c5 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:41.598226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 872/1720] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:41.609717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e9c69-c5af-4e39-b036-cbf434a0c971 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:41.620707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2a5c4-a688-4717-b238-ccb8e7a04e48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:41.622417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2a5c4-a688-4717-b238-ccb8e7a04e48 service swift] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:41.622895 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2a5c4-a688-4717-b238-ccb8e7a04e48 service swift] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:41.623389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2a5c4-a688-4717-b238-ccb8e7a04e48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:41.810876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2a5c4-a688-4717-b238-ccb8e7a04e48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:41.812724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2a5c4-a688-4717-b238-ccb8e7a04e48 service swift] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:41.856275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 849/1721] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:36:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:36:41.876223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e0546-ee37-4c2a-bfb1-0e2dee8187c5 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:41.878142 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-969e0546-ee37-4c2a-bfb1-0e2dee8187c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:41.921810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e0546-ee37-4c2a-bfb1-0e2dee8187c5 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:41.953657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fc689-f590-4567-91aa-293798adab86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:41.955612 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fc689-f590-4567-91aa-293798adab86 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:41.955612 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fc689-f590-4567-91aa-293798adab86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:41.955612 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fc689-f590-4567-91aa-293798adab86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:42.342437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7fc689-f590-4567-91aa-293798adab86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:42.344307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7fc689-f590-4567-91aa-293798adab86 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:42.394603 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 873/1722] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:43.440889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2a5c4-a688-4717-b238-ccb8e7a04e48 service swift] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:43.443146 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a2f2a5c4-a688-4717-b238-ccb8e7a04e48 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:43.702615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2a5c4-a688-4717-b238-ccb8e7a04e48 service swift] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:43.711434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e0a0f-1f0f-4227-9fa1-9546fa5345a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:43.712271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e0a0f-1f0f-4227-9fa1-9546fa5345a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:43.712553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e0a0f-1f0f-4227-9fa1-9546fa5345a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:43.712927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e0a0f-1f0f-4227-9fa1-9546fa5345a1 admin admin] PATH_INFO: `/v3/users/d1c15e66eaea440aadca0373a803eed9` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:43.714186 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e0a0f-1f0f-4227-9fa1-9546fa5345a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c15e66eaea440aadca0373a803eed9)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:43.716548 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e0a0f-1f0f-4227-9fa1-9546fa5345a1 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:43.851133 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c45e0a0f-1f0f-4227-9fa1-9546fa5345a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c15e66eaea440aadca0373a803eed9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:43.889349 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c45e0a0f-1f0f-4227-9fa1-9546fa5345a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c15e66eaea440aadca0373a803eed9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:43.913509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c45e0a0f-1f0f-4227-9fa1-9546fa5345a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c15e66eaea440aadca0373a803eed9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:43.929583 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c45e0a0f-1f0f-4227-9fa1-9546fa5345a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c15e66eaea440aadca0373a803eed9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:43.933644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c45e0a0f-1f0f-4227-9fa1-9546fa5345a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c15e66eaea440aadca0373a803eed9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:43.949913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 850/1723] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:43 2022] DELETE /identity/v3/users/d1c15e66eaea440aadca0373a803eed9 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:43.952470 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d34d1b-ec3a-48ef-b8f5-0c81b246b96f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:43.953009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d34d1b-ec3a-48ef-b8f5-0c81b246b96f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:43.953009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d34d1b-ec3a-48ef-b8f5-0c81b246b96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:43.985596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f3d34d1b-ec3a-48ef-b8f5-0c81b246b96f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:43.987028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f3d34d1b-ec3a-48ef-b8f5-0c81b246b96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:44.110457 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8f503-0443-4c24-b013-a7f18da3da27 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:44.110789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8f503-0443-4c24-b013-a7f18da3da27 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:44.110997 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8f503-0443-4c24-b013-a7f18da3da27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:44.157937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-a5b8f503-0443-4c24-b013-a7f18da3da27 None None] MFA Rules not processed for user `0718a95189434a5ea901c40bc868bea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:44.158984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a5b8f503-0443-4c24-b013-a7f18da3da27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:44.165429 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:44.258064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 874/1724] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:44.288119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d34d1b-ec3a-48ef-b8f5-0c81b246b96f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:44.289817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f3d34d1b-ec3a-48ef-b8f5-0c81b246b96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:44.303247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 851/1725] 10.222.0.74 () {58 vars in 1015 bytes} [Thu Sep 1 16:36:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:44.356735 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:44.430348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d34d1b-ec3a-48ef-b8f5-0c81b246b96f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:44.439227 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f44491-3473-46b1-8e6b-50b12f0d978b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:44.440096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f44491-3473-46b1-8e6b-50b12f0d978b service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:44.440511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f44491-3473-46b1-8e6b-50b12f0d978b service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:44.440874 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f44491-3473-46b1-8e6b-50b12f0d978b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:44.529282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f44491-3473-46b1-8e6b-50b12f0d978b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:44.558141 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f44491-3473-46b1-8e6b-50b12f0d978b service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:44.612633 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 875/1726] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:36:45.233779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8f503-0443-4c24-b013-a7f18da3da27 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:45.236267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a5b8f503-0443-4c24-b013-a7f18da3da27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:45.292601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8f503-0443-4c24-b013-a7f18da3da27 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:45.299549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefe755-dc40-4040-82aa-17ad8f48d522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:45.300499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefe755-dc40-4040-82aa-17ad8f48d522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:45.300852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefe755-dc40-4040-82aa-17ad8f48d522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:45.301109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefe755-dc40-4040-82aa-17ad8f48d522 admin admin] PATH_INFO: `/v3/projects/b2fc62de7a9140f88a87961dde2d445f` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:45.302233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefe755-dc40-4040-82aa-17ad8f48d522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2fc62de7a9140f88a87961dde2d445f)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:45.304786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefe755-dc40-4040-82aa-17ad8f48d522 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:45.364556 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1fefe755-dc40-4040-82aa-17ad8f48d522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2fc62de7a9140f88a87961dde2d445f', 'request_id': 'req-1fefe755-dc40-4040-82aa-17ad8f48d522'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:45.374604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1fefe755-dc40-4040-82aa-17ad8f48d522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2fc62de7a9140f88a87961dde2d445f', 'request_id': 'req-1fefe755-dc40-4040-82aa-17ad8f48d522'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:45.380976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1fefe755-dc40-4040-82aa-17ad8f48d522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2fc62de7a9140f88a87961dde2d445f', 'request_id': 'req-1fefe755-dc40-4040-82aa-17ad8f48d522'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:45.393639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1fefe755-dc40-4040-82aa-17ad8f48d522 admin admin] The token cache is being invalidate because project b2fc62de7a9140f88a87961dde2d445f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:45.394021 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1fefe755-dc40-4040-82aa-17ad8f48d522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:45.394855 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1fefe755-dc40-4040-82aa-17ad8f48d522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:45.396611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 852/1727] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:45 2022] DELETE /identity/v3/projects/b2fc62de7a9140f88a87961dde2d445f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:45.401622 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac242f-e3ea-42cd-af40-39a230f1b4f0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:45.402137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac242f-e3ea-42cd-af40-39a230f1b4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:45.402137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac242f-e3ea-42cd-af40-39a230f1b4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:45.426449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-80ac242f-e3ea-42cd-af40-39a230f1b4f0 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:45.427683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-80ac242f-e3ea-42cd-af40-39a230f1b4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:45.509635 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 876/1728] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:45.523318 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763f69c-2432-4711-815d-8c1cf3afaed0 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:45.523318 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763f69c-2432-4711-815d-8c1cf3afaed0 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:45.523318 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763f69c-2432-4711-815d-8c1cf3afaed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:45.544341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-1763f69c-2432-4711-815d-8c1cf3afaed0 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:45.545849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1763f69c-2432-4711-815d-8c1cf3afaed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:45.554034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 853/1729] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:45.558894 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac242f-e3ea-42cd-af40-39a230f1b4f0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:45.569790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac242f-e3ea-42cd-af40-39a230f1b4f0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:45.574651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4903b-a5ff-4ed9-be6e-841bb4a6bc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:45.575519 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4903b-a5ff-4ed9-be6e-841bb4a6bc46 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:45.575796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4903b-a5ff-4ed9-be6e-841bb4a6bc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:45.576099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4903b-a5ff-4ed9-be6e-841bb4a6bc46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:45.576816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4903b-a5ff-4ed9-be6e-841bb4a6bc46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:45.577137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4903b-a5ff-4ed9-be6e-841bb4a6bc46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:45.579221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4903b-a5ff-4ed9-be6e-841bb4a6bc46 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:45.593342 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 877/1730] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:36:45.598273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763f69c-2432-4711-815d-8c1cf3afaed0 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:45.608538 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763f69c-2432-4711-815d-8c1cf3afaed0 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:45.613219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e02f2a-caee-44e1-afe3-b2c8775ff8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:45.613984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e02f2a-caee-44e1-afe3-b2c8775ff8ef admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:45.614260 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e02f2a-caee-44e1-afe3-b2c8775ff8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:45.614529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e02f2a-caee-44e1-afe3-b2c8775ff8ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:45.615217 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e02f2a-caee-44e1-afe3-b2c8775ff8ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:45.617224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e02f2a-caee-44e1-afe3-b2c8775ff8ef admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:45.648960 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 854/1731] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:45 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:45.653377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4903b-a5ff-4ed9-be6e-841bb4a6bc46 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:45.664816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4903b-a5ff-4ed9-be6e-841bb4a6bc46 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:45.732079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af7454-bc83-4a13-b386-9b71f513a780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:45.732597 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af7454-bc83-4a13-b386-9b71f513a780 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:45.732863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af7454-bc83-4a13-b386-9b71f513a780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:45.733128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af7454-bc83-4a13-b386-9b71f513a780 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:45.733772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af7454-bc83-4a13-b386-9b71f513a780 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:45.735728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af7454-bc83-4a13-b386-9b71f513a780 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:45.771927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 878/1732] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:45 2022] POST /identity/v3/users => generated 282 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:36:45.772910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e02f2a-caee-44e1-afe3-b2c8775ff8ef admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:45.787643 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e02f2a-caee-44e1-afe3-b2c8775ff8ef admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:45.792529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c9b87-b5e0-42a9-92b0-866d3b7b8bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:45.793232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c9b87-b5e0-42a9-92b0-866d3b7b8bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:45.793811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c9b87-b5e0-42a9-92b0-866d3b7b8bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:45.794232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c9b87-b5e0-42a9-92b0-866d3b7b8bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:45.794927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c9b87-b5e0-42a9-92b0-866d3b7b8bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:45.796520 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c9b87-b5e0-42a9-92b0-866d3b7b8bd4 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:45.809208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 855/1733] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:45.812485 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af7454-bc83-4a13-b386-9b71f513a780 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:45.826385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af7454-bc83-4a13-b386-9b71f513a780 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:45.832266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e7788-6e52-4d13-8532-63d6f0942778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:45.833296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e7788-6e52-4d13-8532-63d6f0942778 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:45.833663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e7788-6e52-4d13-8532-63d6f0942778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:45.834026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e7788-6e52-4d13-8532-63d6f0942778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:45.834928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e7788-6e52-4d13-8532-63d6f0942778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:45.836674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e7788-6e52-4d13-8532-63d6f0942778 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:45.851354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 879/1734] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:45.856534 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c9b87-b5e0-42a9-92b0-866d3b7b8bd4 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:45.867685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c9b87-b5e0-42a9-92b0-866d3b7b8bd4 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:45.874458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f75e8c-fc22-4a4c-8346-b7b1c86a8521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:45.875149 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75e8c-fc22-4a4c-8346-b7b1c86a8521 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:45.875430 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75e8c-fc22-4a4c-8346-b7b1c86a8521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:45.875874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75e8c-fc22-4a4c-8346-b7b1c86a8521 admin admin] PATH_INFO: `/v3/projects/a350b122781c49758d315232546b610b/users/f43bf450dfd24e4da356b3d00a0dc99b/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:45.894212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f75e8c-fc22-4a4c-8346-b7b1c86a8521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a350b122781c49758d315232546b610b, user_id=f43bf450dfd24e4da356b3d00a0dc99b, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:45.896994 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f75e8c-fc22-4a4c-8346-b7b1c86a8521 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:45.912475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 856/1735] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:45 2022] PUT /identity/v3/projects/a350b122781c49758d315232546b610b/users/f43bf450dfd24e4da356b3d00a0dc99b/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:36:45.923994 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eab0b-6d0a-4ad6-b7bf-142baefb2520 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:45.924256 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eab0b-6d0a-4ad6-b7bf-142baefb2520 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:45.924529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1eab0b-6d0a-4ad6-b7bf-142baefb2520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:45.966120 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ee1eab0b-6d0a-4ad6-b7bf-142baefb2520 None None] MFA Rules not processed for user `f43bf450dfd24e4da356b3d00a0dc99b`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:45.968021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ee1eab0b-6d0a-4ad6-b7bf-142baefb2520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:45.990758 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:46.043096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 880/1736] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:46.048172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f75e8c-fc22-4a4c-8346-b7b1c86a8521 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:46.095216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:46.130067 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f75e8c-fc22-4a4c-8346-b7b1c86a8521 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:46.138726 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd179b9f-a000-4dde-99c1-3ceb838d902e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:46.139465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd179b9f-a000-4dde-99c1-3ceb838d902e admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:46.139686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd179b9f-a000-4dde-99c1-3ceb838d902e admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:46.140137 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd179b9f-a000-4dde-99c1-3ceb838d902e admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:46.140847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd179b9f-a000-4dde-99c1-3ceb838d902e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:46.142847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd179b9f-a000-4dde-99c1-3ceb838d902e admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:46.174616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 857/1737] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:46 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:46.179290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1eab0b-6d0a-4ad6-b7bf-142baefb2520 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:46.192068 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1eab0b-6d0a-4ad6-b7bf-142baefb2520 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:46.266242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b29cc-f484-4410-bb88-b5a9437375ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:46.267166 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b29cc-f484-4410-bb88-b5a9437375ed admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:46.267511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b29cc-f484-4410-bb88-b5a9437375ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:46.267929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b29cc-f484-4410-bb88-b5a9437375ed admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:46.268702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b29cc-f484-4410-bb88-b5a9437375ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:46.270708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b29cc-f484-4410-bb88-b5a9437375ed admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:46.321533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 881/1738] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:46 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:46.325332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd179b9f-a000-4dde-99c1-3ceb838d902e admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:46.346521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd179b9f-a000-4dde-99c1-3ceb838d902e admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:46.379954 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956d09a-9092-44f9-ae64-21ea8fd25fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:46.381637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956d09a-9092-44f9-ae64-21ea8fd25fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:46.382204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956d09a-9092-44f9-ae64-21ea8fd25fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:46.382485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956d09a-9092-44f9-ae64-21ea8fd25fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:46.403506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956d09a-9092-44f9-ae64-21ea8fd25fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:46.412314 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956d09a-9092-44f9-ae64-21ea8fd25fe3 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:46.480567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b29cc-f484-4410-bb88-b5a9437375ed admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:46.488683 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 858/1739] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:46.520160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b29cc-f484-4410-bb88-b5a9437375ed admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:46.548201 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc74af-56b0-403a-97ee-67bb4cfa642c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:46.549713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc74af-56b0-403a-97ee-67bb4cfa642c admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:46.550556 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc74af-56b0-403a-97ee-67bb4cfa642c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:46.551461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc74af-56b0-403a-97ee-67bb4cfa642c admin admin] PATH_INFO: `/v3/projects/4c9dae3232b74eb18b196f95c5c7bf85/users/1dec91d1f8b2483b9f7a20a0fe630940/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:46.577810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc74af-56b0-403a-97ee-67bb4cfa642c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c9dae3232b74eb18b196f95c5c7bf85, user_id=1dec91d1f8b2483b9f7a20a0fe630940, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:46.581274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc74af-56b0-403a-97ee-67bb4cfa642c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:46.629628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 882/1740] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:46 2022] PUT /identity/v3/projects/4c9dae3232b74eb18b196f95c5c7bf85/users/1dec91d1f8b2483b9f7a20a0fe630940/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:46.633971 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956d09a-9092-44f9-ae64-21ea8fd25fe3 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:46.647400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956d09a-9092-44f9-ae64-21ea8fd25fe3 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:46.840458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb0a96-51fc-4e88-920a-d3f3707c0f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:46.841110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb0a96-51fc-4e88-920a-d3f3707c0f26 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:46.841494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb0a96-51fc-4e88-920a-d3f3707c0f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:46.841651 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb0a96-51fc-4e88-920a-d3f3707c0f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:46.842302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb0a96-51fc-4e88-920a-d3f3707c0f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:46.862058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb0a96-51fc-4e88-920a-d3f3707c0f26 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:46.903549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 859/1741] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:46.907534 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc74af-56b0-403a-97ee-67bb4cfa642c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:46.924421 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc74af-56b0-403a-97ee-67bb4cfa642c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:46.929439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb1d61-e352-4e45-bc69-37eeb6574dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:46.930177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb1d61-e352-4e45-bc69-37eeb6574dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:46.930463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb1d61-e352-4e45-bc69-37eeb6574dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:46.930735 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb1d61-e352-4e45-bc69-37eeb6574dfe admin admin] PATH_INFO: `/v3/projects/4c9dae3232b74eb18b196f95c5c7bf85/users/1dec91d1f8b2483b9f7a20a0fe630940/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:46.932832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb1d61-e352-4e45-bc69-37eeb6574dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c9dae3232b74eb18b196f95c5c7bf85, user_id=1dec91d1f8b2483b9f7a20a0fe630940, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:46.935794 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb1d61-e352-4e45-bc69-37eeb6574dfe admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:46.948273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 883/1742] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:46 2022] PUT /identity/v3/projects/4c9dae3232b74eb18b196f95c5c7bf85/users/1dec91d1f8b2483b9f7a20a0fe630940/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:46.961725 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d16fce-8aca-44be-a49b-34005c63d925 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:46.962977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d16fce-8aca-44be-a49b-34005c63d925 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:46.963850 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d16fce-8aca-44be-a49b-34005c63d925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:47.038026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-a4d16fce-8aca-44be-a49b-34005c63d925 None None] MFA Rules not processed for user `1dec91d1f8b2483b9f7a20a0fe630940`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:47.046446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a4d16fce-8aca-44be-a49b-34005c63d925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:47.306017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 860/1743] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:47.329709 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb1d61-e352-4e45-bc69-37eeb6574dfe admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:47.331767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-88fb1d61-e352-4e45-bc69-37eeb6574dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:47.502914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb1d61-e352-4e45-bc69-37eeb6574dfe admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:47.514577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33442e31-8284-45ba-b843-9e8f7283a2fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:47.515219 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33442e31-8284-45ba-b843-9e8f7283a2fc service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:47.515496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33442e31-8284-45ba-b843-9e8f7283a2fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:47.515763 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33442e31-8284-45ba-b843-9e8f7283a2fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:47.526651 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d16fce-8aca-44be-a49b-34005c63d925 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:47.528860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a4d16fce-8aca-44be-a49b-34005c63d925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:47.529956 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33442e31-8284-45ba-b843-9e8f7283a2fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:47.531407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33442e31-8284-45ba-b843-9e8f7283a2fc service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:47.542160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 884/1744] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:47.586311 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:47.643813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d16fce-8aca-44be-a49b-34005c63d925 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:47.648575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ab408-e8a4-48fc-a2db-852e9bdd56a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:47.649285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ab408-e8a4-48fc-a2db-852e9bdd56a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:47.650128 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ab408-e8a4-48fc-a2db-852e9bdd56a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:47.650128 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ab408-e8a4-48fc-a2db-852e9bdd56a0 admin admin] PATH_INFO: `/v3/users/b8efd053afdf4550b0b65f043de6e516` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:47.651200 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ab408-e8a4-48fc-a2db-852e9bdd56a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8efd053afdf4550b0b65f043de6e516)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:47.654154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ab408-e8a4-48fc-a2db-852e9bdd56a0 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:47.665546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33442e31-8284-45ba-b843-9e8f7283a2fc service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:47.668051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-33442e31-8284-45ba-b843-9e8f7283a2fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:47.712385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:47.749661 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-713ab408-e8a4-48fc-a2db-852e9bdd56a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8efd053afdf4550b0b65f043de6e516'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:47.784899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-713ab408-e8a4-48fc-a2db-852e9bdd56a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8efd053afdf4550b0b65f043de6e516'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:47.796239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-713ab408-e8a4-48fc-a2db-852e9bdd56a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8efd053afdf4550b0b65f043de6e516'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:47.809100 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-713ab408-e8a4-48fc-a2db-852e9bdd56a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8efd053afdf4550b0b65f043de6e516'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:47.810286 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-713ab408-e8a4-48fc-a2db-852e9bdd56a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8efd053afdf4550b0b65f043de6e516'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:47.811604 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33442e31-8284-45ba-b843-9e8f7283a2fc service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:47.827945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 861/1745] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:47 2022] DELETE /identity/v3/users/b8efd053afdf4550b0b65f043de6e516 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:47.831161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2d542-303e-4290-b500-a658dc786a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:47.831800 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2d542-303e-4290-b500-a658dc786a48 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:47.832119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2d542-303e-4290-b500-a658dc786a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:47.832621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2d542-303e-4290-b500-a658dc786a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:47.836543 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ab408-e8a4-48fc-a2db-852e9bdd56a0 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:47.837244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-96e2d542-303e-4290-b500-a658dc786a48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:47.838978 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-713ab408-e8a4-48fc-a2db-852e9bdd56a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:47.854492 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2d542-303e-4290-b500-a658dc786a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:47.856198 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2d542-303e-4290-b500-a658dc786a48 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:47.865919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 885/1746] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:47.866882 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2d542-303e-4290-b500-a658dc786a48 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:47.868573 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-96e2d542-303e-4290-b500-a658dc786a48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:47.885034 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2d542-303e-4290-b500-a658dc786a48 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:48.013207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5221b6f-0e71-4011-95c9-9cf35e47a8e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:48.014558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5221b6f-0e71-4011-95c9-9cf35e47a8e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:48.014837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5221b6f-0e71-4011-95c9-9cf35e47a8e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:48.015247 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5221b6f-0e71-4011-95c9-9cf35e47a8e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:48.021346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ab408-e8a4-48fc-a2db-852e9bdd56a0 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:48.032521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5221b6f-0e71-4011-95c9-9cf35e47a8e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:48.034299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57018e3-0146-40b0-aac3-6ef39cbafe43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:48.034702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5221b6f-0e71-4011-95c9-9cf35e47a8e2 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:48.035146 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57018e3-0146-40b0-aac3-6ef39cbafe43 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:48.035535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57018e3-0146-40b0-aac3-6ef39cbafe43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:48.035882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57018e3-0146-40b0-aac3-6ef39cbafe43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:48.061393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 886/1747] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:48.061897 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5221b6f-0e71-4011-95c9-9cf35e47a8e2 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:48.092067 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57018e3-0146-40b0-aac3-6ef39cbafe43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:48.093606 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57018e3-0146-40b0-aac3-6ef39cbafe43 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:48.096091 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5221b6f-0e71-4011-95c9-9cf35e47a8e2 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:48.110023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe794f5-5666-4481-8c7b-dcb4a685599f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:48.112951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe794f5-5666-4481-8c7b-dcb4a685599f service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:48.113238 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe794f5-5666-4481-8c7b-dcb4a685599f service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:48.113570 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe794f5-5666-4481-8c7b-dcb4a685599f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:48.114641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 862/1748] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:48.115359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57018e3-0146-40b0-aac3-6ef39cbafe43 service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:48.137760 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe794f5-5666-4481-8c7b-dcb4a685599f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:48.139496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe794f5-5666-4481-8c7b-dcb4a685599f service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:48.157987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57018e3-0146-40b0-aac3-6ef39cbafe43 service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:48.177260 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34853dbd-69ad-4d07-82bf-cfd12a740c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:48.181443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34853dbd-69ad-4d07-82bf-cfd12a740c4d service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:48.183018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34853dbd-69ad-4d07-82bf-cfd12a740c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:48.183442 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34853dbd-69ad-4d07-82bf-cfd12a740c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:48.188556 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 887/1749] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:48.190619 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe794f5-5666-4481-8c7b-dcb4a685599f service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:48.202311 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe794f5-5666-4481-8c7b-dcb4a685599f service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:48.204078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34853dbd-69ad-4d07-82bf-cfd12a740c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:48.207597 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480c5b4-0112-49c1-ab1a-5505a9260d42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:48.208404 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480c5b4-0112-49c1-ab1a-5505a9260d42 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:48.208683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480c5b4-0112-49c1-ab1a-5505a9260d42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:48.208940 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480c5b4-0112-49c1-ab1a-5505a9260d42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:48.220466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34853dbd-69ad-4d07-82bf-cfd12a740c4d service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:48.221665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480c5b4-0112-49c1-ab1a-5505a9260d42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:48.223245 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480c5b4-0112-49c1-ab1a-5505a9260d42 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:48.252672 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 888/1750] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:48.254275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f4d10-2ed6-486c-a10a-46c515592b8d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:48.254563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f4d10-2ed6-486c-a10a-46c515592b8d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:48.254946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f4d10-2ed6-486c-a10a-46c515592b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:48.258973 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 863/1751] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:48.303280 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-148f4d10-2ed6-486c-a10a-46c515592b8d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:48.305095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-148f4d10-2ed6-486c-a10a-46c515592b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:48.451262 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 889/1752] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:48.483617 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34853dbd-69ad-4d07-82bf-cfd12a740c4d service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:48.494854 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34853dbd-69ad-4d07-82bf-cfd12a740c4d service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:48.508125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8814ad-0c9e-4316-b658-7451559b5df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:48.512584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8814ad-0c9e-4316-b658-7451559b5df6 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:48.514067 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8814ad-0c9e-4316-b658-7451559b5df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:48.514416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8814ad-0c9e-4316-b658-7451559b5df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:48.523207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f4d10-2ed6-486c-a10a-46c515592b8d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:48.524824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-148f4d10-2ed6-486c-a10a-46c515592b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:48.536417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8814ad-0c9e-4316-b658-7451559b5df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:48.541243 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8814ad-0c9e-4316-b658-7451559b5df6 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:48.554363 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f4d10-2ed6-486c-a10a-46c515592b8d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:48.567583 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 864/1753] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:48.569036 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c00cd-98e3-473f-ac2f-9c2a7670334a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:48.571550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c00cd-98e3-473f-ac2f-9c2a7670334a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:48.571849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c00cd-98e3-473f-ac2f-9c2a7670334a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:48.572129 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c00cd-98e3-473f-ac2f-9c2a7670334a admin admin] PATH_INFO: `/v3/users/f43bf450dfd24e4da356b3d00a0dc99b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:48.577074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c00cd-98e3-473f-ac2f-9c2a7670334a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f43bf450dfd24e4da356b3d00a0dc99b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:48.581567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c00cd-98e3-473f-ac2f-9c2a7670334a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:48.685075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:48.711593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9b7c00cd-98e3-473f-ac2f-9c2a7670334a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f43bf450dfd24e4da356b3d00a0dc99b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:48.722881 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9b7c00cd-98e3-473f-ac2f-9c2a7670334a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f43bf450dfd24e4da356b3d00a0dc99b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:48.723944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9b7c00cd-98e3-473f-ac2f-9c2a7670334a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f43bf450dfd24e4da356b3d00a0dc99b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:48.743158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9b7c00cd-98e3-473f-ac2f-9c2a7670334a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f43bf450dfd24e4da356b3d00a0dc99b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:48.756568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9b7c00cd-98e3-473f-ac2f-9c2a7670334a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f43bf450dfd24e4da356b3d00a0dc99b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:48.764727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 890/1754] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:48 2022] DELETE /identity/v3/users/f43bf450dfd24e4da356b3d00a0dc99b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:48.769410 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80623700-5e77-4a48-baba-883def978f46 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:48.769973 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80623700-5e77-4a48-baba-883def978f46 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:48.770442 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80623700-5e77-4a48-baba-883def978f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:48.806658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-80623700-5e77-4a48-baba-883def978f46 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:48.810754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-80623700-5e77-4a48-baba-883def978f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:48.872024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:48.969793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 865/1755] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:49.055557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c00cd-98e3-473f-ac2f-9c2a7670334a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:49.057918 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9b7c00cd-98e3-473f-ac2f-9c2a7670334a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:49.202760 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c00cd-98e3-473f-ac2f-9c2a7670334a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:49.208017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e18ee-d0a0-44cf-b711-cc9e9d326b4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:49.208915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e18ee-d0a0-44cf-b711-cc9e9d326b4f service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:49.209402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e18ee-d0a0-44cf-b711-cc9e9d326b4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:49.210953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e18ee-d0a0-44cf-b711-cc9e9d326b4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:49.223626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e18ee-d0a0-44cf-b711-cc9e9d326b4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:49.225823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e18ee-d0a0-44cf-b711-cc9e9d326b4f service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:49.240338 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 891/1756] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:50.253873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80623700-5e77-4a48-baba-883def978f46 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:50.255637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-80623700-5e77-4a48-baba-883def978f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:50.282063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80623700-5e77-4a48-baba-883def978f46 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:50.287509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744241cb-ba07-4416-a19b-ecfa8a390043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:50.288201 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744241cb-ba07-4416-a19b-ecfa8a390043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:50.288615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744241cb-ba07-4416-a19b-ecfa8a390043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:50.288894 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744241cb-ba07-4416-a19b-ecfa8a390043 admin admin] PATH_INFO: `/v3/projects/c194e18f6f6f400594ea75a61fa58427` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:50.297918 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744241cb-ba07-4416-a19b-ecfa8a390043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c194e18f6f6f400594ea75a61fa58427)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:50.299880 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744241cb-ba07-4416-a19b-ecfa8a390043 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:50.412505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-744241cb-ba07-4416-a19b-ecfa8a390043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c194e18f6f6f400594ea75a61fa58427', 'request_id': 'req-744241cb-ba07-4416-a19b-ecfa8a390043'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:50.426252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-744241cb-ba07-4416-a19b-ecfa8a390043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c194e18f6f6f400594ea75a61fa58427', 'request_id': 'req-744241cb-ba07-4416-a19b-ecfa8a390043'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:50.434233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-744241cb-ba07-4416-a19b-ecfa8a390043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c194e18f6f6f400594ea75a61fa58427', 'request_id': 'req-744241cb-ba07-4416-a19b-ecfa8a390043'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:50.469796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-744241cb-ba07-4416-a19b-ecfa8a390043 admin admin] The token cache is being invalidate because project c194e18f6f6f400594ea75a61fa58427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:50.472504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-744241cb-ba07-4416-a19b-ecfa8a390043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:50.476466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-744241cb-ba07-4416-a19b-ecfa8a390043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:50.478402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 866/1757] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:50 2022] DELETE /identity/v3/projects/c194e18f6f6f400594ea75a61fa58427 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:50.492509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e18ee-d0a0-44cf-b711-cc9e9d326b4f service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:50.499784 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ef8df-4fd2-4181-9e40-908a7d532d75 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:50.500138 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ef8df-4fd2-4181-9e40-908a7d532d75 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:50.500630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ef8df-4fd2-4181-9e40-908a7d532d75 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:50.503939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 867/1758] 10.222.0.62 () {54 vars in 921 bytes} [Thu Sep 1 16:36:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:36:50.543785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e18ee-d0a0-44cf-b711-cc9e9d326b4f service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:50.611425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960414d-6cfa-4c14-aeba-bc5f10320ca3 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:50.611709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960414d-6cfa-4c14-aeba-bc5f10320ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:50.612044 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960414d-6cfa-4c14-aeba-bc5f10320ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:50.674507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e960414d-6cfa-4c14-aeba-bc5f10320ca3 None None] MFA Rules not processed for user `2912c02ff6db4cbc97c80cd923238136`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:50.675961 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61c285-f01b-4dfd-ae51-0cbd1d236322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:50.677645 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61c285-f01b-4dfd-ae51-0cbd1d236322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:50.678891 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e960414d-6cfa-4c14-aeba-bc5f10320ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:50.679387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61c285-f01b-4dfd-ae51-0cbd1d236322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:50.680024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61c285-f01b-4dfd-ae51-0cbd1d236322 admin admin] PATH_INFO: `/v3/users/397219fe731d40ddbbac2170ff10f38a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:50.681737 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61c285-f01b-4dfd-ae51-0cbd1d236322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=397219fe731d40ddbbac2170ff10f38a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:50.685264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61c285-f01b-4dfd-ae51-0cbd1d236322 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:50.787047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4f61c285-f01b-4dfd-ae51-0cbd1d236322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '397219fe731d40ddbbac2170ff10f38a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:50.794593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:50.810284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 868/1759] 10.222.0.62 () {58 vars in 1018 bytes} [Thu Sep 1 16:36:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:50.817191 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4f61c285-f01b-4dfd-ae51-0cbd1d236322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '397219fe731d40ddbbac2170ff10f38a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:50.818185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4f61c285-f01b-4dfd-ae51-0cbd1d236322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '397219fe731d40ddbbac2170ff10f38a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:50.836813 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4f61c285-f01b-4dfd-ae51-0cbd1d236322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '397219fe731d40ddbbac2170ff10f38a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:50.846397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4f61c285-f01b-4dfd-ae51-0cbd1d236322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '397219fe731d40ddbbac2170ff10f38a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:50.865298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 892/1760] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:50 2022] DELETE /identity/v3/users/397219fe731d40ddbbac2170ff10f38a => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:51.320689 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960414d-6cfa-4c14-aeba-bc5f10320ca3 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:51.322794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e960414d-6cfa-4c14-aeba-bc5f10320ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:51.346869 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960414d-6cfa-4c14-aeba-bc5f10320ca3 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:51.412809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:51.433602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:51.435004 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:51.435482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:51.435849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] PATH_INFO: `/v3/projects/8677bc0d96ed461d96cd50e46b4807bf` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:51.437419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8677bc0d96ed461d96cd50e46b4807bf)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:51.439760 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:51.446056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61c285-f01b-4dfd-ae51-0cbd1d236322 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:51.447665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4f61c285-f01b-4dfd-ae51-0cbd1d236322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:51.494883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61c285-f01b-4dfd-ae51-0cbd1d236322 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:51.507173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a75c9-0f33-4575-a132-554687be1cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:51.508146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a75c9-0f33-4575-a132-554687be1cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:51.508568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a75c9-0f33-4575-a132-554687be1cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:51.508981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a75c9-0f33-4575-a132-554687be1cd9 admin admin] PATH_INFO: `/v3/projects/a350b122781c49758d315232546b610b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:51.510432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a75c9-0f33-4575-a132-554687be1cd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a350b122781c49758d315232546b610b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:51.513480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a75c9-0f33-4575-a132-554687be1cd9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:51.582476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8677bc0d96ed461d96cd50e46b4807bf', 'request_id': 'req-7c24599e-752a-44c5-a751-1e64b4874d6c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.594450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b57a75c9-0f33-4575-a132-554687be1cd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a350b122781c49758d315232546b610b', 'request_id': 'req-b57a75c9-0f33-4575-a132-554687be1cd9'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.600306 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8677bc0d96ed461d96cd50e46b4807bf', 'request_id': 'req-7c24599e-752a-44c5-a751-1e64b4874d6c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.608473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b57a75c9-0f33-4575-a132-554687be1cd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a350b122781c49758d315232546b610b', 'request_id': 'req-b57a75c9-0f33-4575-a132-554687be1cd9'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.612275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8677bc0d96ed461d96cd50e46b4807bf', 'request_id': 'req-7c24599e-752a-44c5-a751-1e64b4874d6c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.628062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b57a75c9-0f33-4575-a132-554687be1cd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a350b122781c49758d315232546b610b', 'request_id': 'req-b57a75c9-0f33-4575-a132-554687be1cd9'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.638356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] The token cache is being invalidate because project 8677bc0d96ed461d96cd50e46b4807bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:51.638560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.639415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.647225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 869/1761] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:51 2022] DELETE /identity/v3/projects/8677bc0d96ed461d96cd50e46b4807bf => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:51.649920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:51.653422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b57a75c9-0f33-4575-a132-554687be1cd9 admin admin] The token cache is being invalidate because project a350b122781c49758d315232546b610b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:51.654579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:51.655828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b57a75c9-0f33-4575-a132-554687be1cd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.656542 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b57a75c9-0f33-4575-a132-554687be1cd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.661054 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 893/1762] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:51 2022] DELETE /identity/v3/projects/a350b122781c49758d315232546b610b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:51.664350 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a4ac6-32fc-4938-9976-1c72bc9dadb0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:51.664635 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a4ac6-32fc-4938-9976-1c72bc9dadb0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:51.664909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a4ac6-32fc-4938-9976-1c72bc9dadb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:51.723846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f48a4ac6-32fc-4938-9976-1c72bc9dadb0 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:51.724992 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f48a4ac6-32fc-4938-9976-1c72bc9dadb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:51.811161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 894/1763] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:51.812263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a4ac6-32fc-4938-9976-1c72bc9dadb0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:51.813987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f48a4ac6-32fc-4938-9976-1c72bc9dadb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:51.835559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a4ac6-32fc-4938-9976-1c72bc9dadb0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:51.841617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eca80c-bb36-4c17-8887-11643f8ecdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:51.842153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eca80c-bb36-4c17-8887-11643f8ecdb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:51.842607 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eca80c-bb36-4c17-8887-11643f8ecdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:51.842976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eca80c-bb36-4c17-8887-11643f8ecdb0 admin admin] PATH_INFO: `/v3/users/1dec91d1f8b2483b9f7a20a0fe630940` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:51.844305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eca80c-bb36-4c17-8887-11643f8ecdb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dec91d1f8b2483b9f7a20a0fe630940)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:51.846513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eca80c-bb36-4c17-8887-11643f8ecdb0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:51.869268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24599e-752a-44c5-a751-1e64b4874d6c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:51.914042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-33eca80c-bb36-4c17-8887-11643f8ecdb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dec91d1f8b2483b9f7a20a0fe630940'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.918016 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:51.926425 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-33eca80c-bb36-4c17-8887-11643f8ecdb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dec91d1f8b2483b9f7a20a0fe630940'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.927253 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-33eca80c-bb36-4c17-8887-11643f8ecdb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dec91d1f8b2483b9f7a20a0fe630940'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.950569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-33eca80c-bb36-4c17-8887-11643f8ecdb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dec91d1f8b2483b9f7a20a0fe630940'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.960609 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-33eca80c-bb36-4c17-8887-11643f8ecdb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dec91d1f8b2483b9f7a20a0fe630940'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:51.974026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 895/1764] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:51 2022] DELETE /identity/v3/users/1dec91d1f8b2483b9f7a20a0fe630940 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:51.974026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a44e7d-6f3d-42ba-9c71-e04c3f9d1de1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:51.974026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a44e7d-6f3d-42ba-9c71-e04c3f9d1de1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:51.974026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a44e7d-6f3d-42ba-9c71-e04c3f9d1de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:52.016835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a1a44e7d-6f3d-42ba-9c71-e04c3f9d1de1 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:52.018327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a1a44e7d-6f3d-42ba-9c71-e04c3f9d1de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:52.044538 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389acb6f-e341-41cb-ad01-7d470b70bcb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:52.045165 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389acb6f-e341-41cb-ad01-7d470b70bcb1 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:52.046087 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389acb6f-e341-41cb-ad01-7d470b70bcb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:52.046373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389acb6f-e341-41cb-ad01-7d470b70bcb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:52.048307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-389acb6f-e341-41cb-ad01-7d470b70bcb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:52.063979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389acb6f-e341-41cb-ad01-7d470b70bcb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:52.085241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389acb6f-e341-41cb-ad01-7d470b70bcb1 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:52.118725 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:52.149660 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 870/1765] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:52.172266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 896/1766] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:52.177334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389acb6f-e341-41cb-ad01-7d470b70bcb1 service nova] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:52.204751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389acb6f-e341-41cb-ad01-7d470b70bcb1 service nova] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:52.221058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115f5d3-3797-43a3-a9c3-54573d901d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:52.222102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115f5d3-3797-43a3-a9c3-54573d901d7d admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:52.222754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115f5d3-3797-43a3-a9c3-54573d901d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:52.223417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115f5d3-3797-43a3-a9c3-54573d901d7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:52.249471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115f5d3-3797-43a3-a9c3-54573d901d7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:52.250624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115f5d3-3797-43a3-a9c3-54573d901d7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:52.252950 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115f5d3-3797-43a3-a9c3-54573d901d7d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:52.298735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 871/1767] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:36:52.301097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a44e7d-6f3d-42ba-9c71-e04c3f9d1de1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:52.318205 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a44e7d-6f3d-42ba-9c71-e04c3f9d1de1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:52.327199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c3b39-bab8-4b86-bee8-162ae9b16514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:52.328126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c3b39-bab8-4b86-bee8-162ae9b16514 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:52.328409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c3b39-bab8-4b86-bee8-162ae9b16514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:52.328991 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c3b39-bab8-4b86-bee8-162ae9b16514 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:52.329880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c3b39-bab8-4b86-bee8-162ae9b16514 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:52.331909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c3b39-bab8-4b86-bee8-162ae9b16514 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:52.394247 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 897/1768] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:52 2022] POST /identity/v3/projects => generated 377 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:36:52.395907 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115f5d3-3797-43a3-a9c3-54573d901d7d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:52.417944 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115f5d3-3797-43a3-a9c3-54573d901d7d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:52.572603 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003a0aa-c1e2-4d77-b40b-c8147eaccf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:52.573233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003a0aa-c1e2-4d77-b40b-c8147eaccf1e admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:52.573508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003a0aa-c1e2-4d77-b40b-c8147eaccf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:52.573861 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003a0aa-c1e2-4d77-b40b-c8147eaccf1e admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:52.574421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003a0aa-c1e2-4d77-b40b-c8147eaccf1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:52.576409 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003a0aa-c1e2-4d77-b40b-c8147eaccf1e admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:52.578559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c3b39-bab8-4b86-bee8-162ae9b16514 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:52.580935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-799c3b39-bab8-4b86-bee8-162ae9b16514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:52.606586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 872/1769] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:52 2022] POST /identity/v3/users => generated 289 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:52.608566 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003a0aa-c1e2-4d77-b40b-c8147eaccf1e admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:52.619475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c3b39-bab8-4b86-bee8-162ae9b16514 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:52.644690 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d722bd-6800-4992-b70b-a9ed16cb1da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:52.645936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d722bd-6800-4992-b70b-a9ed16cb1da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:52.646246 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d722bd-6800-4992-b70b-a9ed16cb1da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:52.646483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d722bd-6800-4992-b70b-a9ed16cb1da0 admin admin] PATH_INFO: `/v3/projects/4c9dae3232b74eb18b196f95c5c7bf85` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:52.647682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d722bd-6800-4992-b70b-a9ed16cb1da0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c9dae3232b74eb18b196f95c5c7bf85)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:52.650063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003a0aa-c1e2-4d77-b40b-c8147eaccf1e admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:52.650692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d722bd-6800-4992-b70b-a9ed16cb1da0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:52.680703 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab675f9f-5901-4acc-880a-8afac7910348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:52.681363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab675f9f-5901-4acc-880a-8afac7910348 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:52.681635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab675f9f-5901-4acc-880a-8afac7910348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:52.681863 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab675f9f-5901-4acc-880a-8afac7910348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:52.682610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab675f9f-5901-4acc-880a-8afac7910348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:52.684563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab675f9f-5901-4acc-880a-8afac7910348 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:52.724906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 873/1770] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:52.729723 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab675f9f-5901-4acc-880a-8afac7910348 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:52.747699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab675f9f-5901-4acc-880a-8afac7910348 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:52.754827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c6d722bd-6800-4992-b70b-a9ed16cb1da0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c9dae3232b74eb18b196f95c5c7bf85', 'request_id': 'req-c6d722bd-6800-4992-b70b-a9ed16cb1da0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:52.757064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c6d722bd-6800-4992-b70b-a9ed16cb1da0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c9dae3232b74eb18b196f95c5c7bf85', 'request_id': 'req-c6d722bd-6800-4992-b70b-a9ed16cb1da0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:52.767549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c6d722bd-6800-4992-b70b-a9ed16cb1da0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c9dae3232b74eb18b196f95c5c7bf85', 'request_id': 'req-c6d722bd-6800-4992-b70b-a9ed16cb1da0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:52.787965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c6d722bd-6800-4992-b70b-a9ed16cb1da0 admin admin] The token cache is being invalidate because project 4c9dae3232b74eb18b196f95c5c7bf85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:52.788524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c6d722bd-6800-4992-b70b-a9ed16cb1da0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:52.789313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c6d722bd-6800-4992-b70b-a9ed16cb1da0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:52.795229 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 898/1771] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:52 2022] DELETE /identity/v3/projects/4c9dae3232b74eb18b196f95c5c7bf85 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:36:52.798748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb17b9-c002-41e3-b8b2-7eec46e35ca5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:52.799393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb17b9-c002-41e3-b8b2-7eec46e35ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:52.799765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb17b9-c002-41e3-b8b2-7eec46e35ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:52.827320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65077374-aedf-402b-b6fb-879c334599b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:52.827956 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65077374-aedf-402b-b6fb-879c334599b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:52.828298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65077374-aedf-402b-b6fb-879c334599b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:52.828563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65077374-aedf-402b-b6fb-879c334599b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:52.829660 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65077374-aedf-402b-b6fb-879c334599b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:52.831505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65077374-aedf-402b-b6fb-879c334599b2 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:52.845466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 874/1772] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:52.846778 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-14fb17b9-c002-41e3-b8b2-7eec46e35ca5 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:52.848133 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-14fb17b9-c002-41e3-b8b2-7eec46e35ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:52.850279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65077374-aedf-402b-b6fb-879c334599b2 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:52.852533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-65077374-aedf-402b-b6fb-879c334599b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:52.872234 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 899/1773] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:52.880303 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2002b3-3337-4050-ab0a-871bfc6eccb1 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:52.880583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2002b3-3337-4050-ab0a-871bfc6eccb1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:52.880989 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2002b3-3337-4050-ab0a-871bfc6eccb1 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:52.881194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65077374-aedf-402b-b6fb-879c334599b2 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:52.882463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 900/1774] 10.222.0.62 () {54 vars in 921 bytes} [Thu Sep 1 16:36:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:36:52.888115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c780c-42cd-448d-981c-c885823d292e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:52.888880 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c780c-42cd-448d-981c-c885823d292e admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:52.889224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c780c-42cd-448d-981c-c885823d292e admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:52.889562 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c780c-42cd-448d-981c-c885823d292e admin admin] PATH_INFO: `/v3/projects/666110d65d5b4bef9ea5fff7f162b44b/users/2fa1d5dd5ad7459ab7954baaf686612b/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:52.892536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e03a22-7e3b-4902-9a9c-00e5dc11b189 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:52.893431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e03a22-7e3b-4902-9a9c-00e5dc11b189 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:52.894311 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e03a22-7e3b-4902-9a9c-00e5dc11b189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:52.928068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c780c-42cd-448d-981c-c885823d292e admin admin] RBAC: Authorizing `identity:create_grant(project_id=666110d65d5b4bef9ea5fff7f162b44b, user_id=2fa1d5dd5ad7459ab7954baaf686612b, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:52.931390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c780c-42cd-448d-981c-c885823d292e admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:52.949859 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-92e03a22-7e3b-4902-9a9c-00e5dc11b189 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:52.951196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-92e03a22-7e3b-4902-9a9c-00e5dc11b189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:52.969184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 901/1776] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:52.969376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 875/1776] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:52 2022] PUT /identity/v3/projects/666110d65d5b4bef9ea5fff7f162b44b/users/2fa1d5dd5ad7459ab7954baaf686612b/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:36:52.970211 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c780c-42cd-448d-981c-c885823d292e admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:52.970911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bebf2-97ef-4d76-aa27-3b9b477e02ef None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:52.971195 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bebf2-97ef-4d76-aa27-3b9b477e02ef None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:52.971465 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bebf2-97ef-4d76-aa27-3b9b477e02ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:53.002694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cf3bebf2-97ef-4d76-aa27-3b9b477e02ef None None] MFA Rules not processed for user `2912c02ff6db4cbc97c80cd923238136`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:53.004420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cf3bebf2-97ef-4d76-aa27-3b9b477e02ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:53.019446 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:INSERT=3,keystone:DELETE=10,keystone:SELECT=87 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:53.046222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:53.106829 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c780c-42cd-448d-981c-c885823d292e admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:53.120493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77789cfe-51e6-4744-b7db-076df2b5f953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:53.122460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77789cfe-51e6-4744-b7db-076df2b5f953 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:53.123084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77789cfe-51e6-4744-b7db-076df2b5f953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:53.123653 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77789cfe-51e6-4744-b7db-076df2b5f953 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:53.124686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77789cfe-51e6-4744-b7db-076df2b5f953 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:36:53.126242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77789cfe-51e6-4744-b7db-076df2b5f953 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:53.128169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77789cfe-51e6-4744-b7db-076df2b5f953 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:53.137415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 902/1777] 10.222.0.62 () {58 vars in 1018 bytes} [Thu Sep 1 16:36:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:53.139205 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a93f41-da03-46e0-8df7-01f2c45ba36f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:53.139514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a93f41-da03-46e0-8df7-01f2c45ba36f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:53.139786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a93f41-da03-46e0-8df7-01f2c45ba36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:53.145441 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 876/1778] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:36:53.149987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77789cfe-51e6-4744-b7db-076df2b5f953 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:53.162399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77789cfe-51e6-4744-b7db-076df2b5f953 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:53.168482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf2f26-6e2b-43eb-816c-9923afb541b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:53.169400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf2f26-6e2b-43eb-816c-9923afb541b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:53.169681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf2f26-6e2b-43eb-816c-9923afb541b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:53.169952 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf2f26-6e2b-43eb-816c-9923afb541b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:53.170516 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b0a93f41-da03-46e0-8df7-01f2c45ba36f None None] MFA Rules not processed for user `2fa1d5dd5ad7459ab7954baaf686612b`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:53.170628 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf2f26-6e2b-43eb-816c-9923afb541b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:53.171695 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b0a93f41-da03-46e0-8df7-01f2c45ba36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:53.172548 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf2f26-6e2b-43eb-816c-9923afb541b7 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:53.209161 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 877/1779] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:53 2022] POST /identity/v3/projects => generated 363 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:36:53.215271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf2f26-6e2b-43eb-816c-9923afb541b7 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:53.313905 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf2f26-6e2b-43eb-816c-9923afb541b7 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:53.326027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0a7c0-e24b-4b8d-9a34-defd7ead1232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:53.326027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0a7c0-e24b-4b8d-9a34-defd7ead1232 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:53.326332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 903/1780] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:53.326419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0a7c0-e24b-4b8d-9a34-defd7ead1232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:53.326419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0a7c0-e24b-4b8d-9a34-defd7ead1232 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:53.326419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0a7c0-e24b-4b8d-9a34-defd7ead1232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:53.327483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0a7c0-e24b-4b8d-9a34-defd7ead1232 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:53.328234 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a93f41-da03-46e0-8df7-01f2c45ba36f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:53.360540 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 878/1781] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:53 2022] POST /identity/v3/users => generated 282 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:53.364745 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0a7c0-e24b-4b8d-9a34-defd7ead1232 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:53.369292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a93f41-da03-46e0-8df7-01f2c45ba36f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:53.378112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5ca53-e4ee-4491-9da9-1f12f56619fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:53.384223 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0a7c0-e24b-4b8d-9a34-defd7ead1232 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:53.388197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5ca53-e4ee-4491-9da9-1f12f56619fa admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:53.388485 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5ca53-e4ee-4491-9da9-1f12f56619fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:53.389291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5ca53-e4ee-4491-9da9-1f12f56619fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:53.389977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c5ca53-e4ee-4491-9da9-1f12f56619fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:53.391278 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12177deb-5202-4dbd-be4f-d1056719e25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:53.392481 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12177deb-5202-4dbd-be4f-d1056719e25f admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:53.393216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12177deb-5202-4dbd-be4f-d1056719e25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:53.393609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12177deb-5202-4dbd-be4f-d1056719e25f admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:53.394233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12177deb-5202-4dbd-be4f-d1056719e25f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:53.395780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c5ca53-e4ee-4491-9da9-1f12f56619fa admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:53.395914 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12177deb-5202-4dbd-be4f-d1056719e25f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:53.410086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 879/1782] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:53.414133 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12177deb-5202-4dbd-be4f-d1056719e25f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:53.440574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12177deb-5202-4dbd-be4f-d1056719e25f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:53.454734 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893ce64-dce2-4954-babd-47c7916b4d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:53.455572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893ce64-dce2-4954-babd-47c7916b4d34 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:53.455853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893ce64-dce2-4954-babd-47c7916b4d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:53.456799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893ce64-dce2-4954-babd-47c7916b4d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:53.457633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893ce64-dce2-4954-babd-47c7916b4d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:53.459389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893ce64-dce2-4954-babd-47c7916b4d34 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:53.474676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 880/1783] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:53.479445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893ce64-dce2-4954-babd-47c7916b4d34 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:53.497666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 904/1784] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:53 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:36:53.498628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5ca53-e4ee-4491-9da9-1f12f56619fa admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:53.515401 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893ce64-dce2-4954-babd-47c7916b4d34 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:53.603015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ea64b-00e4-4b96-9bfa-ac0eee68d16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:53.604054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ea64b-00e4-4b96-9bfa-ac0eee68d16b admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:53.604434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ea64b-00e4-4b96-9bfa-ac0eee68d16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:53.604805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ea64b-00e4-4b96-9bfa-ac0eee68d16b admin admin] PATH_INFO: `/v3/projects/92d50e8f59164496bf75d7275128942f/users/0a9d1bc94ba747c0aa0451c92da3e27b/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:53.613074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5ca53-e4ee-4491-9da9-1f12f56619fa admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:53.618197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d059524-ffee-47ec-8587-d9f7bfba718d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:53.619059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d059524-ffee-47ec-8587-d9f7bfba718d admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:53.619488 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d059524-ffee-47ec-8587-d9f7bfba718d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:53.619913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d059524-ffee-47ec-8587-d9f7bfba718d admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:53.620831 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d059524-ffee-47ec-8587-d9f7bfba718d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:53.622916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d059524-ffee-47ec-8587-d9f7bfba718d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:53.631133 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ea64b-00e4-4b96-9bfa-ac0eee68d16b admin admin] RBAC: Authorizing `identity:create_grant(project_id=92d50e8f59164496bf75d7275128942f, user_id=0a9d1bc94ba747c0aa0451c92da3e27b, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:53.634682 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ea64b-00e4-4b96-9bfa-ac0eee68d16b admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:53.647552 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:53.658466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 881/1785] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:53 2022] PUT /identity/v3/projects/92d50e8f59164496bf75d7275128942f/users/0a9d1bc94ba747c0aa0451c92da3e27b/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:53.670713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b864f-e010-4902-81f3-dbe7a654d875 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:53.671004 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b864f-e010-4902-81f3-dbe7a654d875 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:53.671631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b864f-e010-4902-81f3-dbe7a654d875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:53.676920 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 905/1786] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:53 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:53.681912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d059524-ffee-47ec-8587-d9f7bfba718d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:53.691598 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:53.704370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d059524-ffee-47ec-8587-d9f7bfba718d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:53.718230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-327b864f-e010-4902-81f3-dbe7a654d875 None None] MFA Rules not processed for user `0a9d1bc94ba747c0aa0451c92da3e27b`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:53.719791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-327b864f-e010-4902-81f3-dbe7a654d875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:53.861333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 882/1787] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:53.862279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b864f-e010-4902-81f3-dbe7a654d875 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:53.871681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84f0de-b742-45dc-a0b7-d0e448cbc24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:53.872921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84f0de-b742-45dc-a0b7-d0e448cbc24d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:53.873207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84f0de-b742-45dc-a0b7-d0e448cbc24d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:53.873473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84f0de-b742-45dc-a0b7-d0e448cbc24d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:53.874144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84f0de-b742-45dc-a0b7-d0e448cbc24d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:53.875771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84f0de-b742-45dc-a0b7-d0e448cbc24d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:53.885821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b864f-e010-4902-81f3-dbe7a654d875 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:53.893192 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07349673-5b3c-4830-8be8-80cc3d44d463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:53.893937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07349673-5b3c-4830-8be8-80cc3d44d463 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:53.894222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07349673-5b3c-4830-8be8-80cc3d44d463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:53.894573 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07349673-5b3c-4830-8be8-80cc3d44d463 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:53.895219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07349673-5b3c-4830-8be8-80cc3d44d463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:53.897532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 906/1788] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:36:53.899249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84f0de-b742-45dc-a0b7-d0e448cbc24d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:53.900458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07349673-5b3c-4830-8be8-80cc3d44d463 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:53.903054 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3e84f0de-b742-45dc-a0b7-d0e448cbc24d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:53.972427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 883/1789] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:53 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:53.973632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07349673-5b3c-4830-8be8-80cc3d44d463 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:54.065528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84f0de-b742-45dc-a0b7-d0e448cbc24d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:54.117270 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07349673-5b3c-4830-8be8-80cc3d44d463 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:54.146433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84c0eb-2983-4980-9cbc-6feb67ce5dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:54.147590 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84c0eb-2983-4980-9cbc-6feb67ce5dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:54.148046 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84c0eb-2983-4980-9cbc-6feb67ce5dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:54.148459 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84c0eb-2983-4980-9cbc-6feb67ce5dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:54.149344 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84c0eb-2983-4980-9cbc-6feb67ce5dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:54.151384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84c0eb-2983-4980-9cbc-6feb67ce5dc4 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:54.169331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 884/1790] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:54.169331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84c0eb-2983-4980-9cbc-6feb67ce5dc4 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:54.183515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84c0eb-2983-4980-9cbc-6feb67ce5dc4 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:54.191109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcfe5c-b6ab-415b-bbb6-ed62d42337c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:54.191979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcfe5c-b6ab-415b-bbb6-ed62d42337c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:54.192419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcfe5c-b6ab-415b-bbb6-ed62d42337c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:54.192870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcfe5c-b6ab-415b-bbb6-ed62d42337c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:54.193846 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcfe5c-b6ab-415b-bbb6-ed62d42337c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:54.196330 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcfe5c-b6ab-415b-bbb6-ed62d42337c6 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:54.201461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b5d93-d123-4c23-9053-b613a64b30da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:54.202294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b5d93-d123-4c23-9053-b613a64b30da service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:54.202604 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b5d93-d123-4c23-9053-b613a64b30da service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:54.203097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b5d93-d123-4c23-9053-b613a64b30da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:54.228836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b5d93-d123-4c23-9053-b613a64b30da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:54.233297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b5d93-d123-4c23-9053-b613a64b30da service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:54.235595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 885/1791] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:54 2022] POST /identity/v3/users => generated 289 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:54.237970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcfe5c-b6ab-415b-bbb6-ed62d42337c6 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:54.251101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcfe5c-b6ab-415b-bbb6-ed62d42337c6 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:54.256826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde8d05-8bd3-432c-aa5e-07103f802892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:54.257659 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde8d05-8bd3-432c-aa5e-07103f802892 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:54.258107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde8d05-8bd3-432c-aa5e-07103f802892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:54.258466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde8d05-8bd3-432c-aa5e-07103f802892 admin admin] PATH_INFO: `/v3/projects/e2f3915849c040279fae7b1d0b99c133/users/135bd75da2d841339a2b39f0c054ac62/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:54.267262 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 907/1792] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:54.272240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b5d93-d123-4c23-9053-b613a64b30da service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:54.293862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde8d05-8bd3-432c-aa5e-07103f802892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f3915849c040279fae7b1d0b99c133, user_id=135bd75da2d841339a2b39f0c054ac62, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:54.294855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b5d93-d123-4c23-9053-b613a64b30da service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:54.297349 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde8d05-8bd3-432c-aa5e-07103f802892 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:54.301004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5de38-3896-48bd-9972-081cc500603e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:54.302036 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5de38-3896-48bd-9972-081cc500603e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:54.302666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5de38-3896-48bd-9972-081cc500603e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:54.303011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5de38-3896-48bd-9972-081cc500603e admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:54.303802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5de38-3896-48bd-9972-081cc500603e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:54.306757 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5de38-3896-48bd-9972-081cc500603e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:54.348954 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 886/1793] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:54 2022] PUT /identity/v3/projects/e2f3915849c040279fae7b1d0b99c133/users/135bd75da2d841339a2b39f0c054ac62/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:54.351630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45d44-17c0-4159-9cb8-2972a8a57fd2 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:54.351916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 908/1794] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:54.353401 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45d44-17c0-4159-9cb8-2972a8a57fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:54.353801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a45d44-17c0-4159-9cb8-2972a8a57fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:54.356875 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5de38-3896-48bd-9972-081cc500603e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:54.391151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5de38-3896-48bd-9972-081cc500603e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:54.402085 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:54.437069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-75a45d44-17c0-4159-9cb8-2972a8a57fd2 None None] MFA Rules not processed for user `135bd75da2d841339a2b39f0c054ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:54.438526 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-75a45d44-17c0-4159-9cb8-2972a8a57fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:54.452451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:54.507411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba46e79-59aa-4e19-82de-eabd106e3c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:54.508120 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba46e79-59aa-4e19-82de-eabd106e3c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:54.508724 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba46e79-59aa-4e19-82de-eabd106e3c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:54.508724 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba46e79-59aa-4e19-82de-eabd106e3c55 admin admin] PATH_INFO: `/v3/projects/cb6b9eb49f344683aeb59cca283eb782/users/2c001e6fa82e47ab8348878d11c49d08/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:54.541139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba46e79-59aa-4e19-82de-eabd106e3c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6b9eb49f344683aeb59cca283eb782, user_id=2c001e6fa82e47ab8348878d11c49d08, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:54.544200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba46e79-59aa-4e19-82de-eabd106e3c55 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:54.564343 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 887/1795] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:54.568368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a45d44-17c0-4159-9cb8-2972a8a57fd2 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:54.568591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 909/1796] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:54 2022] PUT /identity/v3/projects/cb6b9eb49f344683aeb59cca283eb782/users/2c001e6fa82e47ab8348878d11c49d08/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:54.574068 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba46e79-59aa-4e19-82de-eabd106e3c55 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:54.583730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a45d44-17c0-4159-9cb8-2972a8a57fd2 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:54.686768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090417b-c465-4f7f-974c-542c554f1ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:54.688736 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090417b-c465-4f7f-974c-542c554f1ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:54.689084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090417b-c465-4f7f-974c-542c554f1ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:54.689773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090417b-c465-4f7f-974c-542c554f1ea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:54.690185 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090417b-c465-4f7f-974c-542c554f1ea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:54.692392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090417b-c465-4f7f-974c-542c554f1ea1 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:54.699617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba46e79-59aa-4e19-82de-eabd106e3c55 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:54.706298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33434b3b-d353-4c3f-b376-75de49624824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:54.707405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33434b3b-d353-4c3f-b376-75de49624824 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:54.707959 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33434b3b-d353-4c3f-b376-75de49624824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:54.708079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33434b3b-d353-4c3f-b376-75de49624824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:54.709028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33434b3b-d353-4c3f-b376-75de49624824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:54.711609 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33434b3b-d353-4c3f-b376-75de49624824 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:54.725726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 910/1797] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:54.729903 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33434b3b-d353-4c3f-b376-75de49624824 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:54.735925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 888/1798] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:54 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:36:54.745825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33434b3b-d353-4c3f-b376-75de49624824 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:54.747856 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090417b-c465-4f7f-974c-542c554f1ea1 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:54.824771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d76a1-9518-47cd-b23f-c40f59579325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:54.825574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d76a1-9518-47cd-b23f-c40f59579325 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:54.825847 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d76a1-9518-47cd-b23f-c40f59579325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:54.826112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d76a1-9518-47cd-b23f-c40f59579325 admin admin] PATH_INFO: `/v3/projects/cb6b9eb49f344683aeb59cca283eb782/users/2c001e6fa82e47ab8348878d11c49d08/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:54.828287 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d76a1-9518-47cd-b23f-c40f59579325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6b9eb49f344683aeb59cca283eb782, user_id=2c001e6fa82e47ab8348878d11c49d08, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:54.831344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d76a1-9518-47cd-b23f-c40f59579325 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:54.838500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090417b-c465-4f7f-974c-542c554f1ea1 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:54.843940 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 911/1799] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:54 2022] PUT /identity/v3/projects/cb6b9eb49f344683aeb59cca283eb782/users/2c001e6fa82e47ab8348878d11c49d08/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:36:54.855106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be7c48-fc40-4ab8-a480-3377ec076b9d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:54.855347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be7c48-fc40-4ab8-a480-3377ec076b9d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:54.855628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be7c48-fc40-4ab8-a480-3377ec076b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:54.907000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d3be7c48-fc40-4ab8-a480-3377ec076b9d None None] MFA Rules not processed for user `2c001e6fa82e47ab8348878d11c49d08`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:54.910797 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d3be7c48-fc40-4ab8-a480-3377ec076b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:54.948056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:54.953926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:54.976577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb88cc-197a-411f-a0e9-f41037244898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:54.977365 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb88cc-197a-411f-a0e9-f41037244898 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:54.988767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb88cc-197a-411f-a0e9-f41037244898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:54.990389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb88cc-197a-411f-a0e9-f41037244898 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:54.993009 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb88cc-197a-411f-a0e9-f41037244898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:54.999281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb88cc-197a-411f-a0e9-f41037244898 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:55.042622 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 912/1800] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:55.046172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 889/1801] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:54 2022] POST /identity/v3/users => generated 296 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:55.050477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be7c48-fc40-4ab8-a480-3377ec076b9d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:55.057794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb88cc-197a-411f-a0e9-f41037244898 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:55.060671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3cdb88cc-197a-411f-a0e9-f41037244898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:55.065322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be7c48-fc40-4ab8-a480-3377ec076b9d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:55.070585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb1d91-d50b-4e69-8b76-d40cdad58a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:55.071373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb1d91-d50b-4e69-8b76-d40cdad58a38 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:55.071663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb1d91-d50b-4e69-8b76-d40cdad58a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:55.072021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb1d91-d50b-4e69-8b76-d40cdad58a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:55.072713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb1d91-d50b-4e69-8b76-d40cdad58a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:55.075600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb1d91-d50b-4e69-8b76-d40cdad58a38 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:55.092282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 913/1802] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:55.093442 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb1d91-d50b-4e69-8b76-d40cdad58a38 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:55.103531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb1d91-d50b-4e69-8b76-d40cdad58a38 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:55.114997 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9e419-5c63-4df8-b3da-87c0e9071523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:55.115778 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9e419-5c63-4df8-b3da-87c0e9071523 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:55.116424 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9e419-5c63-4df8-b3da-87c0e9071523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:55.116809 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9e419-5c63-4df8-b3da-87c0e9071523 admin admin] PATH_INFO: `/v3/projects/fd7375d6ceab4ec0a9dc5deca81eb002/users/2c148ba84b3c44719daabbc3ce6b5d89/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:55.129057 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb88cc-197a-411f-a0e9-f41037244898 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:55.133048 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039572ed-8f3f-4b97-ae57-b9016f5575a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:55.133812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039572ed-8f3f-4b97-ae57-b9016f5575a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:55.134210 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039572ed-8f3f-4b97-ae57-b9016f5575a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:55.134756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039572ed-8f3f-4b97-ae57-b9016f5575a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:55.137214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9e419-5c63-4df8-b3da-87c0e9071523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7375d6ceab4ec0a9dc5deca81eb002, user_id=2c148ba84b3c44719daabbc3ce6b5d89, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:55.140432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9e419-5c63-4df8-b3da-87c0e9071523 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:55.146884 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039572ed-8f3f-4b97-ae57-b9016f5575a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:55.148878 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039572ed-8f3f-4b97-ae57-b9016f5575a2 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:55.157893 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 914/1803] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:55 2022] PUT /identity/v3/projects/fd7375d6ceab4ec0a9dc5deca81eb002/users/2c148ba84b3c44719daabbc3ce6b5d89/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:55.160117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 890/1804] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:36:55.161738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9e419-5c63-4df8-b3da-87c0e9071523 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:55.257957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039572ed-8f3f-4b97-ae57-b9016f5575a2 service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:55.258790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9e419-5c63-4df8-b3da-87c0e9071523 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:55.264611 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62f75b-b742-448d-a2de-46853e63ce0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:55.265368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62f75b-b742-448d-a2de-46853e63ce0a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:55.265750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62f75b-b742-448d-a2de-46853e63ce0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:55.266611 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62f75b-b742-448d-a2de-46853e63ce0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:55.266730 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62f75b-b742-448d-a2de-46853e63ce0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:55.268561 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62f75b-b742-448d-a2de-46853e63ce0a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:55.271495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039572ed-8f3f-4b97-ae57-b9016f5575a2 service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:55.280930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 915/1805] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:55.285270 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62f75b-b742-448d-a2de-46853e63ce0a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:55.296647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62f75b-b742-448d-a2de-46853e63ce0a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:55.302436 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ca6d3-b681-4554-9e60-00b4387b3b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:55.303353 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ca6d3-b681-4554-9e60-00b4387b3b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:55.303823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ca6d3-b681-4554-9e60-00b4387b3b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:55.304314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ca6d3-b681-4554-9e60-00b4387b3b0b admin admin] PATH_INFO: `/v3/projects/fd7375d6ceab4ec0a9dc5deca81eb002/users/2c148ba84b3c44719daabbc3ce6b5d89/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:55.306515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ca6d3-b681-4554-9e60-00b4387b3b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7375d6ceab4ec0a9dc5deca81eb002, user_id=2c148ba84b3c44719daabbc3ce6b5d89, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:55.310018 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ca6d3-b681-4554-9e60-00b4387b3b0b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:55.323462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 916/1806] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:55 2022] PUT /identity/v3/projects/fd7375d6ceab4ec0a9dc5deca81eb002/users/2c148ba84b3c44719daabbc3ce6b5d89/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:55.335118 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc825ca-8f36-492b-b719-968dda353925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:55.336376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc825ca-8f36-492b-b719-968dda353925 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:55.336932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc825ca-8f36-492b-b719-968dda353925 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:55.337408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc825ca-8f36-492b-b719-968dda353925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:55.338292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9a903-570c-4c44-b355-b54c0ddd8aea None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:55.338592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9a903-570c-4c44-b355-b54c0ddd8aea None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:55.338875 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9a903-570c-4c44-b355-b54c0ddd8aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:55.396061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-72c9a903-570c-4c44-b355-b54c0ddd8aea None None] MFA Rules not processed for user `2c148ba84b3c44719daabbc3ce6b5d89`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:55.397370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-72c9a903-570c-4c44-b355-b54c0ddd8aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:55.453901 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc825ca-8f36-492b-b719-968dda353925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:55.457934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc825ca-8f36-492b-b719-968dda353925 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:55.468939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:55.473097 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 891/1807] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:55.504322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 917/1808] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:55.516254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc825ca-8f36-492b-b719-968dda353925 service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:55.518611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-4fc825ca-8f36-492b-b719-968dda353925 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:55.541454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc825ca-8f36-492b-b719-968dda353925 service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:55.583441 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:55.629288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812816fd-24ba-4461-bb8a-23f6abab304b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:55.631243 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812816fd-24ba-4461-bb8a-23f6abab304b service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:55.632138 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812816fd-24ba-4461-bb8a-23f6abab304b service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:55.632572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812816fd-24ba-4461-bb8a-23f6abab304b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:55.659029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812816fd-24ba-4461-bb8a-23f6abab304b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:55.661750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812816fd-24ba-4461-bb8a-23f6abab304b service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:55.684496 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 892/1809] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:55.745277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9a903-570c-4c44-b355-b54c0ddd8aea None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:55.758084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9a903-570c-4c44-b355-b54c0ddd8aea None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:55.763128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049735d-8a8c-483c-b828-ad3294e3e972 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:55.766434 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049735d-8a8c-483c-b828-ad3294e3e972 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:55.766773 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049735d-8a8c-483c-b828-ad3294e3e972 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:55.767186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049735d-8a8c-483c-b828-ad3294e3e972 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:55.865526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049735d-8a8c-483c-b828-ad3294e3e972 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:55.867241 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049735d-8a8c-483c-b828-ad3294e3e972 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:55.883173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 918/1810] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:58.379014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812816fd-24ba-4461-bb8a-23f6abab304b service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:58.468860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812816fd-24ba-4461-bb8a-23f6abab304b service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:58.473903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00daf25-0cf4-42f1-a325-372868780c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:58.474555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00daf25-0cf4-42f1-a325-372868780c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:58.474838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00daf25-0cf4-42f1-a325-372868780c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:58.475103 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00daf25-0cf4-42f1-a325-372868780c70 admin admin] PATH_INFO: `/v3/users/0a9d1bc94ba747c0aa0451c92da3e27b` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:58.477142 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00daf25-0cf4-42f1-a325-372868780c70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a9d1bc94ba747c0aa0451c92da3e27b)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:58.478803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00daf25-0cf4-42f1-a325-372868780c70 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:58.571156 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c00daf25-0cf4-42f1-a325-372868780c70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a9d1bc94ba747c0aa0451c92da3e27b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:58.586235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c00daf25-0cf4-42f1-a325-372868780c70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a9d1bc94ba747c0aa0451c92da3e27b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:58.595247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c00daf25-0cf4-42f1-a325-372868780c70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a9d1bc94ba747c0aa0451c92da3e27b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:58.603706 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c00daf25-0cf4-42f1-a325-372868780c70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a9d1bc94ba747c0aa0451c92da3e27b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:58.604451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c00daf25-0cf4-42f1-a325-372868780c70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a9d1bc94ba747c0aa0451c92da3e27b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:58.611693 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 893/1811] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:58 2022] DELETE /identity/v3/users/0a9d1bc94ba747c0aa0451c92da3e27b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:58.616523 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fe757-1b8f-4be4-8af3-a7182c6fc2a5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:58.617428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fe757-1b8f-4be4-8af3-a7182c6fc2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:58.617707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fe757-1b8f-4be4-8af3-a7182c6fc2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:58.639174 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a08fe757-1b8f-4be4-8af3-a7182c6fc2a5 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:58.640463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a08fe757-1b8f-4be4-8af3-a7182c6fc2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:58.715848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 919/1812] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:58.726911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00daf25-0cf4-42f1-a325-372868780c70 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:58.728431 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c00daf25-0cf4-42f1-a325-372868780c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:58.784301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fe757-1b8f-4be4-8af3-a7182c6fc2a5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:58.787425 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a08fe757-1b8f-4be4-8af3-a7182c6fc2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:58.812745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fe757-1b8f-4be4-8af3-a7182c6fc2a5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:58.828362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446f2e3-fce9-464a-ad99-09a0c76ebee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:58.829921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446f2e3-fce9-464a-ad99-09a0c76ebee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:58.830431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446f2e3-fce9-464a-ad99-09a0c76ebee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:58.830885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446f2e3-fce9-464a-ad99-09a0c76ebee9 admin admin] PATH_INFO: `/v3/users/c2a508ae0a1241a89a347348dbca3e7e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:58.833630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446f2e3-fce9-464a-ad99-09a0c76ebee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2a508ae0a1241a89a347348dbca3e7e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:58.836364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446f2e3-fce9-464a-ad99-09a0c76ebee9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:58.854766 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00daf25-0cf4-42f1-a325-372868780c70 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:58.861063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c749fd0-a508-4819-a915-16d485f03a1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:58.862396 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c749fd0-a508-4819-a915-16d485f03a1f service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:58.863004 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c749fd0-a508-4819-a915-16d485f03a1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:58.863510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c749fd0-a508-4819-a915-16d485f03a1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:58.874352 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:58.882061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c749fd0-a508-4819-a915-16d485f03a1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:58.884419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c749fd0-a508-4819-a915-16d485f03a1f service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:58.901071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 894/1813] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:58.917163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b446f2e3-fce9-464a-ad99-09a0c76ebee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2a508ae0a1241a89a347348dbca3e7e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:58.928824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b446f2e3-fce9-464a-ad99-09a0c76ebee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2a508ae0a1241a89a347348dbca3e7e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:58.929709 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b446f2e3-fce9-464a-ad99-09a0c76ebee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2a508ae0a1241a89a347348dbca3e7e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:58.949647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b446f2e3-fce9-464a-ad99-09a0c76ebee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2a508ae0a1241a89a347348dbca3e7e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:58.956872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:58.961779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b446f2e3-fce9-464a-ad99-09a0c76ebee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2a508ae0a1241a89a347348dbca3e7e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:58.986900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 920/1814] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:58 2022] DELETE /identity/v3/users/c2a508ae0a1241a89a347348dbca3e7e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:58.999803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72936b3-7cb3-49bc-a096-28cd5248d735 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:58.999803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72936b3-7cb3-49bc-a096-28cd5248d735 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:58.999803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72936b3-7cb3-49bc-a096-28cd5248d735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:59.019533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-d72936b3-7cb3-49bc-a096-28cd5248d735 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:59.021995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d72936b3-7cb3-49bc-a096-28cd5248d735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:59.108292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 895/1815] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:59.131224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446f2e3-fce9-464a-ad99-09a0c76ebee9 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:59.132728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b446f2e3-fce9-464a-ad99-09a0c76ebee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:59.207653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446f2e3-fce9-464a-ad99-09a0c76ebee9 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:59.212837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad0254-de9a-4be7-8cbf-f69e7cfc8113 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:59.213768 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad0254-de9a-4be7-8cbf-f69e7cfc8113 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:59.214359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad0254-de9a-4be7-8cbf-f69e7cfc8113 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:59.214520 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad0254-de9a-4be7-8cbf-f69e7cfc8113 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:59.232842 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72936b3-7cb3-49bc-a096-28cd5248d735 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:59.234513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d72936b3-7cb3-49bc-a096-28cd5248d735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:59.237866 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad0254-de9a-4be7-8cbf-f69e7cfc8113 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:59.239572 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad0254-de9a-4be7-8cbf-f69e7cfc8113 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:59.247245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72936b3-7cb3-49bc-a096-28cd5248d735 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:59.251184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 921/1816] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:59.251992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5d777-1436-48da-a817-d9e09e23b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:59.252662 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5d777-1436-48da-a817-d9e09e23b431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:59.252942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5d777-1436-48da-a817-d9e09e23b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:59.253214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5d777-1436-48da-a817-d9e09e23b431 admin admin] PATH_INFO: `/v3/users/2fa1d5dd5ad7459ab7954baaf686612b` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:59.254409 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5d777-1436-48da-a817-d9e09e23b431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fa1d5dd5ad7459ab7954baaf686612b)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:59.256381 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5d777-1436-48da-a817-d9e09e23b431 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:59.308506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-adb5d777-1436-48da-a817-d9e09e23b431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fa1d5dd5ad7459ab7954baaf686612b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.322911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-adb5d777-1436-48da-a817-d9e09e23b431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fa1d5dd5ad7459ab7954baaf686612b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.331524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-adb5d777-1436-48da-a817-d9e09e23b431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fa1d5dd5ad7459ab7954baaf686612b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.341358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-adb5d777-1436-48da-a817-d9e09e23b431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fa1d5dd5ad7459ab7954baaf686612b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.342152 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-adb5d777-1436-48da-a817-d9e09e23b431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fa1d5dd5ad7459ab7954baaf686612b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.351353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 896/1817] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:59 2022] DELETE /identity/v3/users/2fa1d5dd5ad7459ab7954baaf686612b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:36:59.356533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b396e-c36b-4ff5-8fb3-721ed5b8a9cf None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:59.356795 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b396e-c36b-4ff5-8fb3-721ed5b8a9cf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:59.357072 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b396e-c36b-4ff5-8fb3-721ed5b8a9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:59.362756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5d777-1436-48da-a817-d9e09e23b431 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:59.364936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-adb5d777-1436-48da-a817-d9e09e23b431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:59.379201 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-358b396e-c36b-4ff5-8fb3-721ed5b8a9cf None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:59.380543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-358b396e-c36b-4ff5-8fb3-721ed5b8a9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:59.448480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5d777-1436-48da-a817-d9e09e23b431 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:59.460000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:59.460667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:59.461025 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:59.461199 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] PATH_INFO: `/v3/projects/92d50e8f59164496bf75d7275128942f` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:59.462348 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=92d50e8f59164496bf75d7275128942f)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:59.464405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:59.472643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 922/1818] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:59.477385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:59.482594 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b396e-c36b-4ff5-8fb3-721ed5b8a9cf None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:59.484812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-358b396e-c36b-4ff5-8fb3-721ed5b8a9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:59.528748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92d50e8f59164496bf75d7275128942f', 'request_id': 'req-8a542230-ee5a-4018-a833-6d5e0df8a5ec'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.540881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92d50e8f59164496bf75d7275128942f', 'request_id': 'req-8a542230-ee5a-4018-a833-6d5e0df8a5ec'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.550621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92d50e8f59164496bf75d7275128942f', 'request_id': 'req-8a542230-ee5a-4018-a833-6d5e0df8a5ec'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.565180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] The token cache is being invalidate because project 92d50e8f59164496bf75d7275128942f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:59.565472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.566421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.568520 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 897/1819] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:59 2022] DELETE /identity/v3/projects/92d50e8f59164496bf75d7275128942f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:59.572252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:59.574097 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:59.574729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b396e-c36b-4ff5-8fb3-721ed5b8a9cf None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:59.740387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c45a48-36e1-44cc-ae71-9e0dadeae2a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:59.741053 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45a48-36e1-44cc-ae71-9e0dadeae2a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:59.741347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45a48-36e1-44cc-ae71-9e0dadeae2a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:59.741489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a542230-ee5a-4018-a833-6d5e0df8a5ec admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:59.741588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45a48-36e1-44cc-ae71-9e0dadeae2a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:59.744922 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a4c45a48-36e1-44cc-ae71-9e0dadeae2a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:59.747279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089c7a2-22e7-4ef4-b563-fd8a40b57f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:59.748874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089c7a2-22e7-4ef4-b563-fd8a40b57f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:59.749149 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089c7a2-22e7-4ef4-b563-fd8a40b57f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:59.749414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089c7a2-22e7-4ef4-b563-fd8a40b57f0f admin admin] PATH_INFO: `/v3/users/2c001e6fa82e47ab8348878d11c49d08` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:59.755068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089c7a2-22e7-4ef4-b563-fd8a40b57f0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c001e6fa82e47ab8348878d11c49d08)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:59.756809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089c7a2-22e7-4ef4-b563-fd8a40b57f0f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:59.772878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:59.775981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c45a48-36e1-44cc-ae71-9e0dadeae2a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:59.777665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c45a48-36e1-44cc-ae71-9e0dadeae2a7 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:59.789844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 923/1820] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:36:59.790379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c45a48-36e1-44cc-ae71-9e0dadeae2a7 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:59.791671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a4c45a48-36e1-44cc-ae71-9e0dadeae2a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:59.804330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c45a48-36e1-44cc-ae71-9e0dadeae2a7 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:59.811584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:59.812391 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:59.812704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:59.813043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] PATH_INFO: `/v3/projects/a8a3cb02a86b43fb9e7f0689bafeae64` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:59.814243 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8a3cb02a86b43fb9e7f0689bafeae64)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:59.816662 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:59.819847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2089c7a2-22e7-4ef4-b563-fd8a40b57f0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c001e6fa82e47ab8348878d11c49d08'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.842367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2089c7a2-22e7-4ef4-b563-fd8a40b57f0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c001e6fa82e47ab8348878d11c49d08'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.863138 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2089c7a2-22e7-4ef4-b563-fd8a40b57f0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c001e6fa82e47ab8348878d11c49d08'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.874930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2089c7a2-22e7-4ef4-b563-fd8a40b57f0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c001e6fa82e47ab8348878d11c49d08'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.875882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2089c7a2-22e7-4ef4-b563-fd8a40b57f0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c001e6fa82e47ab8348878d11c49d08'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.880748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8a3cb02a86b43fb9e7f0689bafeae64', 'request_id': 'req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.884306 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 898/1821] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:59 2022] DELETE /identity/v3/users/2c001e6fa82e47ab8348878d11c49d08 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:36:59.886854 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8a3cb02a86b43fb9e7f0689bafeae64', 'request_id': 'req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.894633 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8a3cb02a86b43fb9e7f0689bafeae64', 'request_id': 'req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.910661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] The token cache is being invalidate because project a8a3cb02a86b43fb9e7f0689bafeae64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:36:59.911165 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.911769 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:36:59.913773 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 924/1822] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:36:59 2022] DELETE /identity/v3/projects/a8a3cb02a86b43fb9e7f0689bafeae64 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:59.917928 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089c7a2-22e7-4ef4-b563-fd8a40b57f0f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:59.920077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2089c7a2-22e7-4ef4-b563-fd8a40b57f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:59.964799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:00.052759 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089c7a2-22e7-4ef4-b563-fd8a40b57f0f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:00.059124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60a1ff-9dce-497f-bfb9-a39585200e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:00.059851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60a1ff-9dce-497f-bfb9-a39585200e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:00.060139 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60a1ff-9dce-497f-bfb9-a39585200e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:00.060412 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60a1ff-9dce-497f-bfb9-a39585200e11 admin admin] PATH_INFO: `/v3/users/e1a6aa0986904ee2b6c288d03e191350` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:00.064506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e60a1ff-9dce-497f-bfb9-a39585200e11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1a6aa0986904ee2b6c288d03e191350)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:00.064506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e60a1ff-9dce-497f-bfb9-a39585200e11 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:00.128179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9e60a1ff-9dce-497f-bfb9-a39585200e11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1a6aa0986904ee2b6c288d03e191350'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.144447 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9e60a1ff-9dce-497f-bfb9-a39585200e11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1a6aa0986904ee2b6c288d03e191350'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.154600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9e60a1ff-9dce-497f-bfb9-a39585200e11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1a6aa0986904ee2b6c288d03e191350'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.168964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9e60a1ff-9dce-497f-bfb9-a39585200e11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1a6aa0986904ee2b6c288d03e191350'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.170269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9e60a1ff-9dce-497f-bfb9-a39585200e11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1a6aa0986904ee2b6c288d03e191350'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.178014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 899/1823] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:36:59 2022] DELETE /identity/v3/users/e1a6aa0986904ee2b6c288d03e191350 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:00.295629 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:00.297435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:00.427640 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5bd05-fbb4-4ebc-9127-ae0f729c8793 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:00.521402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37adf5-16fd-4eef-ae05-4964d91bfefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:00.522492 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37adf5-16fd-4eef-ae05-4964d91bfefb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:00.522774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37adf5-16fd-4eef-ae05-4964d91bfefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:00.523139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37adf5-16fd-4eef-ae05-4964d91bfefb admin admin] PATH_INFO: `/v3/projects/cb6b9eb49f344683aeb59cca283eb782` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:00.525250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37adf5-16fd-4eef-ae05-4964d91bfefb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6b9eb49f344683aeb59cca283eb782)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:00.527231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37adf5-16fd-4eef-ae05-4964d91bfefb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:00.586927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:00.622565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ae37adf5-16fd-4eef-ae05-4964d91bfefb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6b9eb49f344683aeb59cca283eb782', 'request_id': 'req-ae37adf5-16fd-4eef-ae05-4964d91bfefb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.632493 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ae37adf5-16fd-4eef-ae05-4964d91bfefb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6b9eb49f344683aeb59cca283eb782', 'request_id': 'req-ae37adf5-16fd-4eef-ae05-4964d91bfefb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.642032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60a1ff-9dce-497f-bfb9-a39585200e11 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:00.643527 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9e60a1ff-9dce-497f-bfb9-a39585200e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:00.660861 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ae37adf5-16fd-4eef-ae05-4964d91bfefb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6b9eb49f344683aeb59cca283eb782', 'request_id': 'req-ae37adf5-16fd-4eef-ae05-4964d91bfefb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.667508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60a1ff-9dce-497f-bfb9-a39585200e11 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:00.682119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ae37adf5-16fd-4eef-ae05-4964d91bfefb admin admin] The token cache is being invalidate because project cb6b9eb49f344683aeb59cca283eb782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:00.683107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ae37adf5-16fd-4eef-ae05-4964d91bfefb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.684310 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ae37adf5-16fd-4eef-ae05-4964d91bfefb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.686381 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 925/1824] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:00 2022] DELETE /identity/v3/projects/cb6b9eb49f344683aeb59cca283eb782 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:00.705565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37be7d-ab8a-4bf7-9eba-1f1af5a21638 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:00.706144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37be7d-ab8a-4bf7-9eba-1f1af5a21638 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:00.708426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37be7d-ab8a-4bf7-9eba-1f1af5a21638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:00.763207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-fe37be7d-ab8a-4bf7-9eba-1f1af5a21638 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:00.764526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fe37be7d-ab8a-4bf7-9eba-1f1af5a21638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:00.806499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:00.808987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:00.810763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:00.811281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d admin admin] PATH_INFO: `/v3/projects/96c08444b98b4a21b4767b1fe84cd9bb` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:00.813328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=96c08444b98b4a21b4767b1fe84cd9bb)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:00.816552 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:00.888865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 926/1825] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:00.890266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5371a-c7d8-4e67-84f8-0ce2fe353c07 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:00.890558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5371a-c7d8-4e67-84f8-0ce2fe353c07 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:00.893013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5371a-c7d8-4e67-84f8-0ce2fe353c07 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:00.894427 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 927/1826] 10.222.0.62 () {54 vars in 921 bytes} [Thu Sep 1 16:37:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:37:00.898465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:00.898714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5371a-c7d8-4e67-84f8-0ce2fe353c07 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:00.905038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b8f5371a-c7d8-4e67-84f8-0ce2fe353c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:00.918953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96c08444b98b4a21b4767b1fe84cd9bb', 'request_id': 'req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.934470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96c08444b98b4a21b4767b1fe84cd9bb', 'request_id': 'req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.943748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96c08444b98b4a21b4767b1fe84cd9bb', 'request_id': 'req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.960209 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d admin admin] The token cache is being invalidate because project 96c08444b98b4a21b4767b1fe84cd9bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:00.960528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.962439 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ef6ee542-f8aa-437f-9c03-7db48f7c8b4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:00.965239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 900/1827] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:00 2022] DELETE /identity/v3/projects/96c08444b98b4a21b4767b1fe84cd9bb => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:00.971273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e6c27-2838-4158-8704-3eecb4326e68 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:00.971572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e6c27-2838-4158-8704-3eecb4326e68 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:00.973572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e6c27-2838-4158-8704-3eecb4326e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.020890 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-b19e6c27-2838-4158-8704-3eecb4326e68 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:01.023019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b19e6c27-2838-4158-8704-3eecb4326e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:01.030241 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5371a-c7d8-4e67-84f8-0ce2fe353c07 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:01.043827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:01.045233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 901/1828] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:01.045745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.046567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75e72d-bf96-4295-80cb-598a25f97134 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.046854 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75e72d-bf96-4295-80cb-598a25f97134 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.047093 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.047195 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75e72d-bf96-4295-80cb-598a25f97134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.047640 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] PATH_INFO: `/v3/projects/666110d65d5b4bef9ea5fff7f162b44b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.050061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] RBAC: Authorizing `identity:delete_project(project_id=666110d65d5b4bef9ea5fff7f162b44b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:01.053642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:01.080612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6b75e72d-bf96-4295-80cb-598a25f97134 None None] MFA Rules not processed for user `2912c02ff6db4cbc97c80cd923238136`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:01.082039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6b75e72d-bf96-4295-80cb-598a25f97134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:01.132934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '666110d65d5b4bef9ea5fff7f162b44b', 'request_id': 'req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.140650 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '666110d65d5b4bef9ea5fff7f162b44b', 'request_id': 'req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.141777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:01.149693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '666110d65d5b4bef9ea5fff7f162b44b', 'request_id': 'req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.169868 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] The token cache is being invalidate because project 666110d65d5b4bef9ea5fff7f162b44b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:01.170189 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.171111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.175785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 902/1829] 10.222.0.62 () {58 vars in 1018 bytes} [Thu Sep 1 16:37:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:01.175983 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 928/1830] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:00 2022] DELETE /identity/v3/projects/666110d65d5b4bef9ea5fff7f162b44b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:01.175983 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:01.176131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e8b7a-e263-46b0-9861-2d8dc205d618 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.176131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e8b7a-e263-46b0-9861-2d8dc205d618 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.176131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e8b7a-e263-46b0-9861-2d8dc205d618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.176317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:01.209868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-fb7e8b7a-e263-46b0-9861-2d8dc205d618 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:01.212726 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-fb7e8b7a-e263-46b0-9861-2d8dc205d618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:01.263924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ea5c1-1b28-4c79-b22d-4885ab44772d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:01.269043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b5f88-0ad5-4e91-a2a9-d922504daab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:01.270084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b5f88-0ad5-4e91-a2a9-d922504daab1 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.270453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b5f88-0ad5-4e91-a2a9-d922504daab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.270806 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b5f88-0ad5-4e91-a2a9-d922504daab1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.271920 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b5f88-0ad5-4e91-a2a9-d922504daab1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:37:01.272203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b5f88-0ad5-4e91-a2a9-d922504daab1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:01.274220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b5f88-0ad5-4e91-a2a9-d922504daab1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:01.292575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 929/1831] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:37:01.294361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b5f88-0ad5-4e91-a2a9-d922504daab1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:01.296824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c02b5f88-0ad5-4e91-a2a9-d922504daab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:01.299014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 903/1832] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:01.300465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e8b7a-e263-46b0-9861-2d8dc205d618 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:01.302800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-fb7e8b7a-e263-46b0-9861-2d8dc205d618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:01.323105 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b5f88-0ad5-4e91-a2a9-d922504daab1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:01.330446 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84722c2-33aa-4570-806f-a975d3bb0763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:01.331064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84722c2-33aa-4570-806f-a975d3bb0763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.331409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84722c2-33aa-4570-806f-a975d3bb0763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.331783 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84722c2-33aa-4570-806f-a975d3bb0763 admin admin] PATH_INFO: `/v3/users/135bd75da2d841339a2b39f0c054ac62` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.333337 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84722c2-33aa-4570-806f-a975d3bb0763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=135bd75da2d841339a2b39f0c054ac62)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:01.336633 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84722c2-33aa-4570-806f-a975d3bb0763 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:01.390624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:01.406187 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b84722c2-33aa-4570-806f-a975d3bb0763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '135bd75da2d841339a2b39f0c054ac62'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.420590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b84722c2-33aa-4570-806f-a975d3bb0763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '135bd75da2d841339a2b39f0c054ac62'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.421458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b84722c2-33aa-4570-806f-a975d3bb0763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '135bd75da2d841339a2b39f0c054ac62'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.424970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e8b7a-e263-46b0-9861-2d8dc205d618 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:01.432590 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5b9b0-8edb-4b4c-bbdb-c9a9a075e539 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:01.433322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5b9b0-8edb-4b4c-bbdb-c9a9a075e539 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.433771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5b9b0-8edb-4b4c-bbdb-c9a9a075e539 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.434241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5b9b0-8edb-4b4c-bbdb-c9a9a075e539 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.436497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-17d5b9b0-8edb-4b4c-bbdb-c9a9a075e539 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:01.441541 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b84722c2-33aa-4570-806f-a975d3bb0763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '135bd75da2d841339a2b39f0c054ac62'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.451170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b84722c2-33aa-4570-806f-a975d3bb0763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '135bd75da2d841339a2b39f0c054ac62'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.460088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 930/1833] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:01 2022] DELETE /identity/v3/users/135bd75da2d841339a2b39f0c054ac62 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:01.461905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84722c2-33aa-4570-806f-a975d3bb0763 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:01.462310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d5b9b0-8edb-4b4c-bbdb-c9a9a075e539 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:01.468992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d5b9b0-8edb-4b4c-bbdb-c9a9a075e539 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:01.471823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b84722c2-33aa-4570-806f-a975d3bb0763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:01.501093 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 904/1834] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:01.503489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd8d23-ede3-45c4-b5d8-fc98f00d6c01 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.503770 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd8d23-ede3-45c4-b5d8-fc98f00d6c01 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.504068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd8d23-ede3-45c4-b5d8-fc98f00d6c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.549997 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6dbd8d23-ede3-45c4-b5d8-fc98f00d6c01 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:01.551974 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6dbd8d23-ede3-45c4-b5d8-fc98f00d6c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:01.592890 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:01.617430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84722c2-33aa-4570-806f-a975d3bb0763 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:01.631177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88dc61-0a47-40b5-8090-c1a80fc157ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:01.632076 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88dc61-0a47-40b5-8090-c1a80fc157ac admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.632237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88dc61-0a47-40b5-8090-c1a80fc157ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.632483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88dc61-0a47-40b5-8090-c1a80fc157ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.635629 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88dc61-0a47-40b5-8090-c1a80fc157ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:01.639284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88dc61-0a47-40b5-8090-c1a80fc157ac admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:01.701942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 905/1835] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:01.707744 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd8d23-ede3-45c4-b5d8-fc98f00d6c01 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:01.715933 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 931/1836] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:01 2022] POST /identity/v3/projects => generated 385 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:01.717763 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88dc61-0a47-40b5-8090-c1a80fc157ac admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:01.723526 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd8d23-ede3-45c4-b5d8-fc98f00d6c01 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:01.848865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b740b9-8186-4503-8f7c-eafa237cc4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:01.850320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b740b9-8186-4503-8f7c-eafa237cc4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.852124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b740b9-8186-4503-8f7c-eafa237cc4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.852795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b740b9-8186-4503-8f7c-eafa237cc4f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.855316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b740b9-8186-4503-8f7c-eafa237cc4f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:37:01.855637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b740b9-8186-4503-8f7c-eafa237cc4f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:01.860926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b740b9-8186-4503-8f7c-eafa237cc4f6 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:01.888864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 906/1837] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:37:01.892547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b740b9-8186-4503-8f7c-eafa237cc4f6 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:01.892880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88dc61-0a47-40b5-8090-c1a80fc157ac admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:01.908982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9657e5-55c7-491e-ae01-46db67c57a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:01.909614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9657e5-55c7-491e-ae01-46db67c57a97 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.909888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9657e5-55c7-491e-ae01-46db67c57a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.910152 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9657e5-55c7-491e-ae01-46db67c57a97 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.910850 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9657e5-55c7-491e-ae01-46db67c57a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:01.911236 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b740b9-8186-4503-8f7c-eafa237cc4f6 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:01.914144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9657e5-55c7-491e-ae01-46db67c57a97 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:01.917531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a749be4-6318-4310-9954-7e63aa7d0de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:01.918298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a749be4-6318-4310-9954-7e63aa7d0de0 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.919187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a749be4-6318-4310-9954-7e63aa7d0de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.919187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a749be4-6318-4310-9954-7e63aa7d0de0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.919824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a749be4-6318-4310-9954-7e63aa7d0de0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:01.922223 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a749be4-6318-4310-9954-7e63aa7d0de0 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:01.948650 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 932/1838] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:01 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:01.953315 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9657e5-55c7-491e-ae01-46db67c57a97 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:01.962774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 907/1839] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:01 2022] POST /identity/v3/projects => generated 387 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:01.967254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a749be4-6318-4310-9954-7e63aa7d0de0 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:01.968539 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9657e5-55c7-491e-ae01-46db67c57a97 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.025894 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:02.049347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a749be4-6318-4310-9954-7e63aa7d0de0 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.051510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183f47f-5bdb-4006-a181-48205a27d39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.052346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183f47f-5bdb-4006-a181-48205a27d39b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.052730 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183f47f-5bdb-4006-a181-48205a27d39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.053011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183f47f-5bdb-4006-a181-48205a27d39b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.053742 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183f47f-5bdb-4006-a181-48205a27d39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.054805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa8ef8-18b2-45ee-8f4c-6093055109b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.055422 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa8ef8-18b2-45ee-8f4c-6093055109b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.055813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa8ef8-18b2-45ee-8f4c-6093055109b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.056150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183f47f-5bdb-4006-a181-48205a27d39b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.057284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa8ef8-18b2-45ee-8f4c-6093055109b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.058116 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa8ef8-18b2-45ee-8f4c-6093055109b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.060302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa8ef8-18b2-45ee-8f4c-6093055109b8 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.074682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 933/1840] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:02.075718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183f47f-5bdb-4006-a181-48205a27d39b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.077624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3183f47f-5bdb-4006-a181-48205a27d39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:02.099382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183f47f-5bdb-4006-a181-48205a27d39b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.103851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 908/1841] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:01 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:37:02.103851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa8ef8-18b2-45ee-8f4c-6093055109b8 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.105214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.105976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.106343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.106702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] PATH_INFO: `/v3/projects/e2f3915849c040279fae7b1d0b99c133` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.107970 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2f3915849c040279fae7b1d0b99c133)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.110075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.116333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa8ef8-18b2-45ee-8f4c-6093055109b8 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.124333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c54a41-bac9-47a6-8fde-e4103a4f16e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.125701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c54a41-bac9-47a6-8fde-e4103a4f16e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.125986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c54a41-bac9-47a6-8fde-e4103a4f16e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.126577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c54a41-bac9-47a6-8fde-e4103a4f16e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.128349 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c54a41-bac9-47a6-8fde-e4103a4f16e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.129176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c54a41-bac9-47a6-8fde-e4103a4f16e9 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.140832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 909/1842] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:02.144436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c54a41-bac9-47a6-8fde-e4103a4f16e9 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.153208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c54a41-bac9-47a6-8fde-e4103a4f16e9 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.159560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8d327-fb77-49d1-a12f-8adb3b28396a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.160547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d327-fb77-49d1-a12f-8adb3b28396a admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.161005 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d327-fb77-49d1-a12f-8adb3b28396a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.161316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d327-fb77-49d1-a12f-8adb3b28396a admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.162132 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8d327-fb77-49d1-a12f-8adb3b28396a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.163834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8d327-fb77-49d1-a12f-8adb3b28396a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.179059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 910/1843] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:02.179059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8d327-fb77-49d1-a12f-8adb3b28396a admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.179655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2f3915849c040279fae7b1d0b99c133', 'request_id': 'req-5919e2f1-12dd-4f6b-af95-68da5cf81d51'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.186498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2f3915849c040279fae7b1d0b99c133', 'request_id': 'req-5919e2f1-12dd-4f6b-af95-68da5cf81d51'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.187664 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:02.189491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8d327-fb77-49d1-a12f-8adb3b28396a admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.195618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2f3915849c040279fae7b1d0b99c133', 'request_id': 'req-5919e2f1-12dd-4f6b-af95-68da5cf81d51'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.213138 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] The token cache is being invalidate because project e2f3915849c040279fae7b1d0b99c133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:02.214881 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.215460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.217844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 934/1844] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:02 2022] DELETE /identity/v3/projects/e2f3915849c040279fae7b1d0b99c133 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:02.221298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.222793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:02.264977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc7ef0-f6ca-4876-a88f-f8717f1fab1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.265807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc7ef0-f6ca-4876-a88f-f8717f1fab1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.266149 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc7ef0-f6ca-4876-a88f-f8717f1fab1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.266478 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc7ef0-f6ca-4876-a88f-f8717f1fab1d admin admin] PATH_INFO: `/v3/projects/5662a0f1df7943e69e506bdb99da4719/users/63a45eaebf584b62b1a53336e3728060/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.304485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc7ef0-f6ca-4876-a88f-f8717f1fab1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5662a0f1df7943e69e506bdb99da4719, user_id=63a45eaebf584b62b1a53336e3728060, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.310697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc7ef0-f6ca-4876-a88f-f8717f1fab1d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.322260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919e2f1-12dd-4f6b-af95-68da5cf81d51 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.331768 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c15cb-96b3-4108-ab63-b4d76017ec37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.332594 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c15cb-96b3-4108-ab63-b4d76017ec37 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.333100 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c15cb-96b3-4108-ab63-b4d76017ec37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.333217 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c15cb-96b3-4108-ab63-b4d76017ec37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.333978 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c15cb-96b3-4108-ab63-b4d76017ec37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.338024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c15cb-96b3-4108-ab63-b4d76017ec37 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.352995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 911/1845] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:02 2022] PUT /identity/v3/projects/5662a0f1df7943e69e506bdb99da4719/users/63a45eaebf584b62b1a53336e3728060/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:02.356384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc7ef0-f6ca-4876-a88f-f8717f1fab1d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.359866 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c1cc7ef0-f6ca-4876-a88f-f8717f1fab1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:02.365332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 935/1846] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:02.366734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeb45c-0690-4884-9a8a-2062da3734ca None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.367097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeb45c-0690-4884-9a8a-2062da3734ca None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.367621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedeb45c-0690-4884-9a8a-2062da3734ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.406247 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dedeb45c-0690-4884-9a8a-2062da3734ca None None] MFA Rules not processed for user `63a45eaebf584b62b1a53336e3728060`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:02.408333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dedeb45c-0690-4884-9a8a-2062da3734ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:02.491613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc7ef0-f6ca-4876-a88f-f8717f1fab1d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.501844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554db63-ea74-4d51-a4a0-293340f4fa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.502513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554db63-ea74-4d51-a4a0-293340f4fa38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.502788 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554db63-ea74-4d51-a4a0-293340f4fa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.503055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554db63-ea74-4d51-a4a0-293340f4fa38 admin admin] PATH_INFO: `/v3/users/2c148ba84b3c44719daabbc3ce6b5d89` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.504904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554db63-ea74-4d51-a4a0-293340f4fa38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c148ba84b3c44719daabbc3ce6b5d89)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.508514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554db63-ea74-4d51-a4a0-293340f4fa38 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.548383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 936/1847] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:02.550717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedeb45c-0690-4884-9a8a-2062da3734ca None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.569489 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedeb45c-0690-4884-9a8a-2062da3734ca None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.577471 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e9984-f7ec-4510-83d8-aad8991fee6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.579081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e9984-f7ec-4510-83d8-aad8991fee6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.579752 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e9984-f7ec-4510-83d8-aad8991fee6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.580460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e9984-f7ec-4510-83d8-aad8991fee6c admin admin] PATH_INFO: `/v3/projects/d7d49f035a58472ab3d9318a3b46994d/users/f3e845cf12e74809bd8ac6f22fb907bd/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.583268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:02.595785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7554db63-ea74-4d51-a4a0-293340f4fa38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c148ba84b3c44719daabbc3ce6b5d89'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.606154 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e9984-f7ec-4510-83d8-aad8991fee6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d49f035a58472ab3d9318a3b46994d, user_id=f3e845cf12e74809bd8ac6f22fb907bd, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.609347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e9984-f7ec-4510-83d8-aad8991fee6c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.620938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7554db63-ea74-4d51-a4a0-293340f4fa38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c148ba84b3c44719daabbc3ce6b5d89'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.630044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 937/1848] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:02 2022] PUT /identity/v3/projects/d7d49f035a58472ab3d9318a3b46994d/users/f3e845cf12e74809bd8ac6f22fb907bd/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:02.631231 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7554db63-ea74-4d51-a4a0-293340f4fa38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c148ba84b3c44719daabbc3ce6b5d89'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.631575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e9984-f7ec-4510-83d8-aad8991fee6c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.641178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-227e9984-f7ec-4510-83d8-aad8991fee6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:02.642690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7554db63-ea74-4d51-a4a0-293340f4fa38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c148ba84b3c44719daabbc3ce6b5d89'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.643390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7554db63-ea74-4d51-a4a0-293340f4fa38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c148ba84b3c44719daabbc3ce6b5d89'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.653636 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 912/1849] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:02 2022] DELETE /identity/v3/users/2c148ba84b3c44719daabbc3ce6b5d89 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:02.657868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f0762-954f-4ada-a820-33badd2810b3 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.658199 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f0762-954f-4ada-a820-33badd2810b3 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.659330 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f0762-954f-4ada-a820-33badd2810b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.738259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-369f0762-954f-4ada-a820-33badd2810b3 None None] MFA Rules not processed for user `f3e845cf12e74809bd8ac6f22fb907bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:02.740921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-369f0762-954f-4ada-a820-33badd2810b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:02.790414 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:02.822205 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e9984-f7ec-4510-83d8-aad8991fee6c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.872751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 913/1850] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:02.876763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f0762-954f-4ada-a820-33badd2810b3 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.878701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-369f0762-954f-4ada-a820-33badd2810b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:02.952118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b67054-1610-4e85-b017-688899516ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.952933 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b67054-1610-4e85-b017-688899516ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.954730 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b67054-1610-4e85-b017-688899516ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.955902 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b67054-1610-4e85-b017-688899516ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.958332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b67054-1610-4e85-b017-688899516ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.960369 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b67054-1610-4e85-b017-688899516ff2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.994061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f0762-954f-4ada-a820-33badd2810b3 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.005773 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 938/1851] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:02 2022] POST /identity/v3/projects => generated 385 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:03.006413 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9a3cd-3dbb-4dda-94d1-b4b44186fa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.007276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9a3cd-3dbb-4dda-94d1-b4b44186fa5d admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.007935 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9a3cd-3dbb-4dda-94d1-b4b44186fa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.008063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9a3cd-3dbb-4dda-94d1-b4b44186fa5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.008611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9a3cd-3dbb-4dda-94d1-b4b44186fa5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.010845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b67054-1610-4e85-b017-688899516ff2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.011284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9a3cd-3dbb-4dda-94d1-b4b44186fa5d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.019406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-57b67054-1610-4e85-b017-688899516ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:03.059138 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 914/1852] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:02 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:03.060403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9a3cd-3dbb-4dda-94d1-b4b44186fa5d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.062490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-4ef9a3cd-3dbb-4dda-94d1-b4b44186fa5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:03.107074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:03.112288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b67054-1610-4e85-b017-688899516ff2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.146332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9a3cd-3dbb-4dda-94d1-b4b44186fa5d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.152748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.153553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.153898 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.154232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] PATH_INFO: `/v3/projects/fd7375d6ceab4ec0a9dc5deca81eb002` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.155515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7375d6ceab4ec0a9dc5deca81eb002)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.157697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.181996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022456d-a80f-45e9-9f61-02956496daf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.183118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022456d-a80f-45e9-9f61-02956496daf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.183472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022456d-a80f-45e9-9f61-02956496daf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.183808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022456d-a80f-45e9-9f61-02956496daf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.184544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022456d-a80f-45e9-9f61-02956496daf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.187578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022456d-a80f-45e9-9f61-02956496daf2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.212498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 939/1853] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:03 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:03.213466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7375d6ceab4ec0a9dc5deca81eb002', 'request_id': 'req-587864ae-8cc7-4480-a381-50d534f6997e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:03.213627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022456d-a80f-45e9-9f61-02956496daf2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.222794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7375d6ceab4ec0a9dc5deca81eb002', 'request_id': 'req-587864ae-8cc7-4480-a381-50d534f6997e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:03.225491 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022456d-a80f-45e9-9f61-02956496daf2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.229002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7375d6ceab4ec0a9dc5deca81eb002', 'request_id': 'req-587864ae-8cc7-4480-a381-50d534f6997e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:03.243297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] The token cache is being invalidate because project fd7375d6ceab4ec0a9dc5deca81eb002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:03.243297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:03.244598 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:03.244736 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:03.248783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 915/1854] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:03 2022] DELETE /identity/v3/projects/fd7375d6ceab4ec0a9dc5deca81eb002 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:03.249773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.253050 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:03.307486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ca76a-83e5-43e0-88d3-4b924ce60889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.309854 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ca76a-83e5-43e0-88d3-4b924ce60889 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.310134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ca76a-83e5-43e0-88d3-4b924ce60889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.310408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ca76a-83e5-43e0-88d3-4b924ce60889 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.311093 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ca76a-83e5-43e0-88d3-4b924ce60889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.313785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ca76a-83e5-43e0-88d3-4b924ce60889 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.348067 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 940/1855] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:03 2022] POST /identity/v3/users => generated 299 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:03.350023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae2fe5-145d-45cf-a3b3-d436c8d8f3c3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.351900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae2fe5-145d-45cf-a3b3-d436c8d8f3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.352325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae2fe5-145d-45cf-a3b3-d436c8d8f3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.357355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587864ae-8cc7-4480-a381-50d534f6997e admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.363529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ed23c-a308-43d1-83a2-e52e25459a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.364218 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ed23c-a308-43d1-83a2-e52e25459a6c admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.364496 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ed23c-a308-43d1-83a2-e52e25459a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.365512 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ed23c-a308-43d1-83a2-e52e25459a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.365512 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ed23c-a308-43d1-83a2-e52e25459a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.367477 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ed23c-a308-43d1-83a2-e52e25459a6c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.380086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 916/1856] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:03.380086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ed23c-a308-43d1-83a2-e52e25459a6c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.381279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cbae2fe5-145d-45cf-a3b3-d436c8d8f3c3 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:03.382355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-832ed23c-a308-43d1-83a2-e52e25459a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:03.382702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cbae2fe5-145d-45cf-a3b3-d436c8d8f3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:03.393281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 941/1857] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:03.394132 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae2fe5-145d-45cf-a3b3-d436c8d8f3c3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.402379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ed23c-a308-43d1-83a2-e52e25459a6c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.405250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae2fe5-145d-45cf-a3b3-d436c8d8f3c3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.407395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d31b1-edb7-4f42-804e-7945a7bac9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.408066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d31b1-edb7-4f42-804e-7945a7bac9cf admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.408343 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d31b1-edb7-4f42-804e-7945a7bac9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.408610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d31b1-edb7-4f42-804e-7945a7bac9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.409248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d31b1-edb7-4f42-804e-7945a7bac9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.410454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9cfdf-d9bf-467e-a5d2-fb837fb35e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.411126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9cfdf-d9bf-467e-a5d2-fb837fb35e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.411229 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d31b1-edb7-4f42-804e-7945a7bac9cf admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.411508 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9cfdf-d9bf-467e-a5d2-fb837fb35e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.411837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9cfdf-d9bf-467e-a5d2-fb837fb35e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.412783 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b9cfdf-d9bf-467e-a5d2-fb837fb35e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.414596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b9cfdf-d9bf-467e-a5d2-fb837fb35e45 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.423134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 917/1858] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:03.426443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b85ccd-0ba1-4198-bad9-46a0b56a141f None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.426737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b85ccd-0ba1-4198-bad9-46a0b56a141f None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.427015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b85ccd-0ba1-4198-bad9-46a0b56a141f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.439588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 942/1859] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:03.440117 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9cfdf-d9bf-467e-a5d2-fb837fb35e45 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.452149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9cfdf-d9bf-467e-a5d2-fb837fb35e45 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.457203 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-d1b85ccd-0ba1-4198-bad9-46a0b56a141f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:03.457381 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c364aa-96a2-4930-97c3-2483c937cf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.458557 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d1b85ccd-0ba1-4198-bad9-46a0b56a141f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:03.459210 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c364aa-96a2-4930-97c3-2483c937cf6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.459544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c364aa-96a2-4930-97c3-2483c937cf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.459765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c364aa-96a2-4930-97c3-2483c937cf6d admin admin] PATH_INFO: `/v3/projects/10a58e047e3a48ff9adf4c84422fd7ef/users/a4ac97b743d94c45bcb7af639318b669/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.470978 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 918/1860] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:03.470978 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b85ccd-0ba1-4198-bad9-46a0b56a141f None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.479634 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b85ccd-0ba1-4198-bad9-46a0b56a141f None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.482254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c364aa-96a2-4930-97c3-2483c937cf6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a58e047e3a48ff9adf4c84422fd7ef, user_id=a4ac97b743d94c45bcb7af639318b669, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.484935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c364aa-96a2-4930-97c3-2483c937cf6d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.488072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fee7e-b29e-47b3-950e-b7b69d707de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.489342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fee7e-b29e-47b3-950e-b7b69d707de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.489556 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fee7e-b29e-47b3-950e-b7b69d707de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.489930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fee7e-b29e-47b3-950e-b7b69d707de6 admin admin] PATH_INFO: `/v3/projects/6619e7c091814b8d8df313991bf2bfda/users/9cf64ba5ef3d475a954846a863ae04f9/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.500996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 943/1861] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:03 2022] PUT /identity/v3/projects/10a58e047e3a48ff9adf4c84422fd7ef/users/a4ac97b743d94c45bcb7af639318b669/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:03.502052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c364aa-96a2-4930-97c3-2483c937cf6d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.512379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fee7e-b29e-47b3-950e-b7b69d707de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6619e7c091814b8d8df313991bf2bfda, user_id=9cf64ba5ef3d475a954846a863ae04f9, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.515422 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fee7e-b29e-47b3-950e-b7b69d707de6 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.529800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 919/1862] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:03 2022] PUT /identity/v3/projects/6619e7c091814b8d8df313991bf2bfda/users/9cf64ba5ef3d475a954846a863ae04f9/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:03.531355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fee7e-b29e-47b3-950e-b7b69d707de6 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.588603 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:03.596287 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c364aa-96a2-4930-97c3-2483c937cf6d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.646467 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fee7e-b29e-47b3-950e-b7b69d707de6 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.653177 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97839a4b-9bb6-44d7-a80b-dfc2dec21e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.654187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97839a4b-9bb6-44d7-a80b-dfc2dec21e40 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.654956 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97839a4b-9bb6-44d7-a80b-dfc2dec21e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.655143 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97839a4b-9bb6-44d7-a80b-dfc2dec21e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.655955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97839a4b-9bb6-44d7-a80b-dfc2dec21e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.657898 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97839a4b-9bb6-44d7-a80b-dfc2dec21e40 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.673948 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 920/1863] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:03.673948 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130668b-98d4-4e23-8c22-38d1571b3f46 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.673948 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130668b-98d4-4e23-8c22-38d1571b3f46 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.673948 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130668b-98d4-4e23-8c22-38d1571b3f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.675250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244cbfdd-29d4-4abc-ab34-469d3e4adae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.675909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cbfdd-29d4-4abc-ab34-469d3e4adae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.676197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cbfdd-29d4-4abc-ab34-469d3e4adae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.676464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cbfdd-29d4-4abc-ab34-469d3e4adae8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.677234 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cbfdd-29d4-4abc-ab34-469d3e4adae8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:37:03.677409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cbfdd-29d4-4abc-ab34-469d3e4adae8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.679009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cbfdd-29d4-4abc-ab34-469d3e4adae8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.692488 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 944/1864] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:37:03.693577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244cbfdd-29d4-4abc-ab34-469d3e4adae8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.706389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244cbfdd-29d4-4abc-ab34-469d3e4adae8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.712004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2520e5-1039-4c87-95cd-857b99531583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.712595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-2130668b-98d4-4e23-8c22-38d1571b3f46 None None] MFA Rules not processed for user `9cf64ba5ef3d475a954846a863ae04f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:03.712721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2520e5-1039-4c87-95cd-857b99531583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.712936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2520e5-1039-4c87-95cd-857b99531583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.713207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2520e5-1039-4c87-95cd-857b99531583 admin admin] PATH_INFO: `/v3/projects/10a58e047e3a48ff9adf4c84422fd7ef/users/a4ac97b743d94c45bcb7af639318b669/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.714030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2130668b-98d4-4e23-8c22-38d1571b3f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:03.715913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2520e5-1039-4c87-95cd-857b99531583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a58e047e3a48ff9adf4c84422fd7ef, user_id=a4ac97b743d94c45bcb7af639318b669, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.718552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2520e5-1039-4c87-95cd-857b99531583 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.721775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:03.740178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 945/1865] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:03 2022] PUT /identity/v3/projects/10a58e047e3a48ff9adf4c84422fd7ef/users/a4ac97b743d94c45bcb7af639318b669/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:03.742118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2520e5-1039-4c87-95cd-857b99531583 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.787999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 921/1866] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:03.789056 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c81e4-f5f2-45d2-82f4-57179c4dc7d1 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.789337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c81e4-f5f2-45d2-82f4-57179c4dc7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.789605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c81e4-f5f2-45d2-82f4-57179c4dc7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.813542 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2520e5-1039-4c87-95cd-857b99531583 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.819033 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd91f14-7453-45be-9a84-c606aa04a739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.819033 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd91f14-7453-45be-9a84-c606aa04a739 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.819268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd91f14-7453-45be-9a84-c606aa04a739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.819340 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd91f14-7453-45be-9a84-c606aa04a739 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.820001 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd91f14-7453-45be-9a84-c606aa04a739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.821884 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd91f14-7453-45be-9a84-c606aa04a739 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.822328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-597c81e4-f5f2-45d2-82f4-57179c4dc7d1 None None] MFA Rules not processed for user `a4ac97b743d94c45bcb7af639318b669`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:03.823564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-597c81e4-f5f2-45d2-82f4-57179c4dc7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:03.864475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 946/1867] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:03 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:03.865696 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd91f14-7453-45be-9a84-c606aa04a739 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.919339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 922/1868] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:03.920355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c81e4-f5f2-45d2-82f4-57179c4dc7d1 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.931472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c81e4-f5f2-45d2-82f4-57179c4dc7d1 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.937118 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7a1e7-b55d-47f1-8967-c49227f68a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.937390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd91f14-7453-45be-9a84-c606aa04a739 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.937805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7a1e7-b55d-47f1-8967-c49227f68a32 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.938474 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7a1e7-b55d-47f1-8967-c49227f68a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.938474 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7a1e7-b55d-47f1-8967-c49227f68a32 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.938990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7a1e7-b55d-47f1-8967-c49227f68a32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.941164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7a1e7-b55d-47f1-8967-c49227f68a32 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.943149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2465160-13bf-4cf5-b205-b481a459a608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.943761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2465160-13bf-4cf5-b205-b481a459a608 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.944070 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2465160-13bf-4cf5-b205-b481a459a608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.944339 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2465160-13bf-4cf5-b205-b481a459a608 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.944963 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2465160-13bf-4cf5-b205-b481a459a608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.946831 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2465160-13bf-4cf5-b205-b481a459a608 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.967879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 923/1869] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:03 2022] POST /identity/v3/users => generated 293 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:03.968912 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7a1e7-b55d-47f1-8967-c49227f68a32 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.971003 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8ef7a1e7-b55d-47f1-8967-c49227f68a32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:03.984748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 947/1870] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:03 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:03.985768 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2465160-13bf-4cf5-b205-b481a459a608 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.989901 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:04.045785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7a1e7-b55d-47f1-8967-c49227f68a32 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:04.056038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2465160-13bf-4cf5-b205-b481a459a608 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:04.060902 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe84dc3-4817-4cee-b41f-890285b5470f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:04.061546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe84dc3-4817-4cee-b41f-890285b5470f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:04.061816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe84dc3-4817-4cee-b41f-890285b5470f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:04.062089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe84dc3-4817-4cee-b41f-890285b5470f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:04.062745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe84dc3-4817-4cee-b41f-890285b5470f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:04.064402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe84dc3-4817-4cee-b41f-890285b5470f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:04.069693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:04.075668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 948/1871] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:04.076667 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe84dc3-4817-4cee-b41f-890285b5470f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:04.087546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe84dc3-4817-4cee-b41f-890285b5470f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:04.093268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d1b5f-a9dd-46c4-b749-3935de630756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:04.093912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1b5f-a9dd-46c4-b749-3935de630756 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:04.094363 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1b5f-a9dd-46c4-b749-3935de630756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:04.094682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1b5f-a9dd-46c4-b749-3935de630756 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:04.095342 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d1b5f-a9dd-46c4-b749-3935de630756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:04.097376 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d1b5f-a9dd-46c4-b749-3935de630756 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:04.098877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e707b95-7bf0-4e7c-95f7-db8744fde8d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:04.099495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e707b95-7bf0-4e7c-95f7-db8744fde8d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:04.099771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e707b95-7bf0-4e7c-95f7-db8744fde8d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:04.100077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e707b95-7bf0-4e7c-95f7-db8744fde8d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:04.126097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 949/1872] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:04 2022] POST /identity/v3/users => generated 298 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:04.126097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d1b5f-a9dd-46c4-b749-3935de630756 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:04.135713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d1b5f-a9dd-46c4-b749-3935de630756 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:04.140905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034ceba-afec-4aed-86d8-232d4662ac84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:04.141578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034ceba-afec-4aed-86d8-232d4662ac84 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:04.141916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034ceba-afec-4aed-86d8-232d4662ac84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:04.142207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034ceba-afec-4aed-86d8-232d4662ac84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:04.142864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034ceba-afec-4aed-86d8-232d4662ac84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:04.144485 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034ceba-afec-4aed-86d8-232d4662ac84 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:04.155250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 950/1873] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:04.156082 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034ceba-afec-4aed-86d8-232d4662ac84 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:04.165183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e707b95-7bf0-4e7c-95f7-db8744fde8d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:04.165484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034ceba-afec-4aed-86d8-232d4662ac84 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:04.166698 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e707b95-7bf0-4e7c-95f7-db8744fde8d9 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:04.171259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67d2f0-45cc-4302-adb7-8170b4ba8337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:04.171851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67d2f0-45cc-4302-adb7-8170b4ba8337 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:04.172207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67d2f0-45cc-4302-adb7-8170b4ba8337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:04.172479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67d2f0-45cc-4302-adb7-8170b4ba8337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:04.173112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67d2f0-45cc-4302-adb7-8170b4ba8337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:04.174741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67d2f0-45cc-4302-adb7-8170b4ba8337 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:04.176676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 924/1874] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:04.177443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e707b95-7bf0-4e7c-95f7-db8744fde8d9 service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:04.188614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 951/1875] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:04.188751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e707b95-7bf0-4e7c-95f7-db8744fde8d9 service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:04.189745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67d2f0-45cc-4302-adb7-8170b4ba8337 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:04.192435 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e7b46-cf97-4884-99de-e30202e458b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:04.193067 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e7b46-cf97-4884-99de-e30202e458b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:04.193311 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e7b46-cf97-4884-99de-e30202e458b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:04.193599 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e7b46-cf97-4884-99de-e30202e458b9 admin admin] PATH_INFO: `/v3/projects/0423cf0e8bd84438a4913be45146c31d/users/831bd352b5f04686aa352c78f24019e0/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:04.201627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67d2f0-45cc-4302-adb7-8170b4ba8337 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:04.209878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354baa00-5cc9-4e45-85bb-491c0e587e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:04.210580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354baa00-5cc9-4e45-85bb-491c0e587e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:04.210840 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354baa00-5cc9-4e45-85bb-491c0e587e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:04.211108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354baa00-5cc9-4e45-85bb-491c0e587e01 admin admin] PATH_INFO: `/v3/projects/d29ce60144344f6e8b24826f0ed0b4bd/users/c1d10339f4ef4f5b8b415866adcc45e2/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:04.215084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e7b46-cf97-4884-99de-e30202e458b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0423cf0e8bd84438a4913be45146c31d, user_id=831bd352b5f04686aa352c78f24019e0, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:04.217807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e7b46-cf97-4884-99de-e30202e458b9 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:04.228944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354baa00-5cc9-4e45-85bb-491c0e587e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d29ce60144344f6e8b24826f0ed0b4bd, user_id=c1d10339f4ef4f5b8b415866adcc45e2, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:04.232179 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354baa00-5cc9-4e45-85bb-491c0e587e01 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:04.236016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 925/1876] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:04 2022] PUT /identity/v3/projects/0423cf0e8bd84438a4913be45146c31d/users/831bd352b5f04686aa352c78f24019e0/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:04.236984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e7b46-cf97-4884-99de-e30202e458b9 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:04.247611 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 952/1877] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:04 2022] PUT /identity/v3/projects/d29ce60144344f6e8b24826f0ed0b4bd/users/c1d10339f4ef4f5b8b415866adcc45e2/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:04.252398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70cc44-4513-486a-8428-53467d4d3126 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:04.252680 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70cc44-4513-486a-8428-53467d4d3126 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:04.252947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70cc44-4513-486a-8428-53467d4d3126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:04.286689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9f70cc44-4513-486a-8428-53467d4d3126 None None] MFA Rules not processed for user `831bd352b5f04686aa352c78f24019e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:04.288517 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9f70cc44-4513-486a-8428-53467d4d3126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:04.298268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:04.317036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e7b46-cf97-4884-99de-e30202e458b9 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:04.424854 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 953/1878] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:04.424854 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70cc44-4513-486a-8428-53467d4d3126 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:04.444582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23fe1b-206e-4856-8b4d-f4264ec04503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:04.445492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23fe1b-206e-4856-8b4d-f4264ec04503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:04.446079 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23fe1b-206e-4856-8b4d-f4264ec04503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:04.446560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23fe1b-206e-4856-8b4d-f4264ec04503 admin admin] PATH_INFO: `/v3/users/f3e845cf12e74809bd8ac6f22fb907bd` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:04.448677 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23fe1b-206e-4856-8b4d-f4264ec04503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3e845cf12e74809bd8ac6f22fb907bd)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:04.451685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23fe1b-206e-4856-8b4d-f4264ec04503 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:04.529252 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70cc44-4513-486a-8428-53467d4d3126 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:04.534668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86badf90-32c9-4fb2-a859-2fa6d117acb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:04.535438 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86badf90-32c9-4fb2-a859-2fa6d117acb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:04.535714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86badf90-32c9-4fb2-a859-2fa6d117acb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:04.536012 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86badf90-32c9-4fb2-a859-2fa6d117acb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:04.536834 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86badf90-32c9-4fb2-a859-2fa6d117acb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:04.539145 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86badf90-32c9-4fb2-a859-2fa6d117acb5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:04.540148 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5f23fe1b-206e-4856-8b4d-f4264ec04503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3e845cf12e74809bd8ac6f22fb907bd'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:04.556516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5f23fe1b-206e-4856-8b4d-f4264ec04503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3e845cf12e74809bd8ac6f22fb907bd'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:04.556765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 954/1879] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:04.557649 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86badf90-32c9-4fb2-a859-2fa6d117acb5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:04.560104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-86badf90-32c9-4fb2-a859-2fa6d117acb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:04.566441 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5f23fe1b-206e-4856-8b4d-f4264ec04503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3e845cf12e74809bd8ac6f22fb907bd'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:04.576855 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5f23fe1b-206e-4856-8b4d-f4264ec04503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3e845cf12e74809bd8ac6f22fb907bd'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:04.577941 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5f23fe1b-206e-4856-8b4d-f4264ec04503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3e845cf12e74809bd8ac6f22fb907bd'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:04.594569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 926/1880] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:04 2022] DELETE /identity/v3/users/f3e845cf12e74809bd8ac6f22fb907bd => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:04.596205 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23fe1b-206e-4856-8b4d-f4264ec04503 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:04.597329 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5f23fe1b-206e-4856-8b4d-f4264ec04503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:04.613161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:04.681828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86badf90-32c9-4fb2-a859-2fa6d117acb5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:04.754128 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23fe1b-206e-4856-8b4d-f4264ec04503 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:04.761667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef449e-a460-424f-b29f-c0451dffd891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:04.763092 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef449e-a460-424f-b29f-c0451dffd891 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:04.763370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef449e-a460-424f-b29f-c0451dffd891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:04.764010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef449e-a460-424f-b29f-c0451dffd891 admin admin] PATH_INFO: `/v3/projects/d29ce60144344f6e8b24826f0ed0b4bd/users/c1d10339f4ef4f5b8b415866adcc45e2/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:04.767176 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ed1cf-d048-4a8e-9f97-9fae37fa6ab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:04.767801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ed1cf-d048-4a8e-9f97-9fae37fa6ab1 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:04.768111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ed1cf-d048-4a8e-9f97-9fae37fa6ab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:04.768380 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ed1cf-d048-4a8e-9f97-9fae37fa6ab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:04.771295 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef449e-a460-424f-b29f-c0451dffd891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d29ce60144344f6e8b24826f0ed0b4bd, user_id=c1d10339f4ef4f5b8b415866adcc45e2, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:04.773998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef449e-a460-424f-b29f-c0451dffd891 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:04.775532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-914ed1cf-d048-4a8e-9f97-9fae37fa6ab1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:04.795949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 927/1881] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:04 2022] PUT /identity/v3/projects/d29ce60144344f6e8b24826f0ed0b4bd/users/c1d10339f4ef4f5b8b415866adcc45e2/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:04.795949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238966f-4bf4-4bb3-81d2-0ade86773d16 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:04.795949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238966f-4bf4-4bb3-81d2-0ade86773d16 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:04.795949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238966f-4bf4-4bb3-81d2-0ade86773d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:04.825533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6238966f-4bf4-4bb3-81d2-0ade86773d16 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:04.829808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6238966f-4bf4-4bb3-81d2-0ade86773d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:04.845111 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:04.871269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ed1cf-d048-4a8e-9f97-9fae37fa6ab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:04.873214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ed1cf-d048-4a8e-9f97-9fae37fa6ab1 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:04.883210 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 955/1882] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:04.884347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f389ec-1680-4410-87c0-1763ead6ed1e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:04.884627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f389ec-1680-4410-87c0-1763ead6ed1e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:04.884949 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f389ec-1680-4410-87c0-1763ead6ed1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:04.938883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d5f389ec-1680-4410-87c0-1763ead6ed1e None None] MFA Rules not processed for user `c1d10339f4ef4f5b8b415866adcc45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:04.940134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d5f389ec-1680-4410-87c0-1763ead6ed1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:04.955150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 928/1883] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:04.962564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238966f-4bf4-4bb3-81d2-0ade86773d16 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:04.967481 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6238966f-4bf4-4bb3-81d2-0ade86773d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:04.991137 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238966f-4bf4-4bb3-81d2-0ade86773d16 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:05.004353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c62bf-1def-4aab-b6c5-714d991f4a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:05.005006 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c62bf-1def-4aab-b6c5-714d991f4a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:05.005409 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c62bf-1def-4aab-b6c5-714d991f4a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:05.005526 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c62bf-1def-4aab-b6c5-714d991f4a9d admin admin] PATH_INFO: `/v3/users/831bd352b5f04686aa352c78f24019e0` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:05.006650 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c62bf-1def-4aab-b6c5-714d991f4a9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=831bd352b5f04686aa352c78f24019e0)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:05.008717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c62bf-1def-4aab-b6c5-714d991f4a9d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:05.107583 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5f9c62bf-1def-4aab-b6c5-714d991f4a9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '831bd352b5f04686aa352c78f24019e0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:05.126988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5f9c62bf-1def-4aab-b6c5-714d991f4a9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '831bd352b5f04686aa352c78f24019e0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:05.141108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 956/1884] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:05.146052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f389ec-1680-4410-87c0-1763ead6ed1e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:05.146052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d5f389ec-1680-4410-87c0-1763ead6ed1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:05.150864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5f9c62bf-1def-4aab-b6c5-714d991f4a9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '831bd352b5f04686aa352c78f24019e0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:05.164150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5f9c62bf-1def-4aab-b6c5-714d991f4a9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '831bd352b5f04686aa352c78f24019e0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:05.165271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5f9c62bf-1def-4aab-b6c5-714d991f4a9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '831bd352b5f04686aa352c78f24019e0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:05.181435 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 929/1885] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:04 2022] DELETE /identity/v3/users/831bd352b5f04686aa352c78f24019e0 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:05.185535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c62bf-1def-4aab-b6c5-714d991f4a9d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:05.185874 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:05.188492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5f9c62bf-1def-4aab-b6c5-714d991f4a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:05.251697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f389ec-1680-4410-87c0-1763ead6ed1e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:05.278619 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c62bf-1def-4aab-b6c5-714d991f4a9d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:05.282895 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882acab1-6f7d-4408-a6a2-15f9235b5933 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:05.283689 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882acab1-6f7d-4408-a6a2-15f9235b5933 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:05.284137 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882acab1-6f7d-4408-a6a2-15f9235b5933 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:05.284665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882acab1-6f7d-4408-a6a2-15f9235b5933 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:05.287164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-882acab1-6f7d-4408-a6a2-15f9235b5933 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:05.327230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f85051-e898-4a59-a564-a6a06442457a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:05.328174 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f85051-e898-4a59-a564-a6a06442457a service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:05.328989 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f85051-e898-4a59-a564-a6a06442457a service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:05.329395 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f85051-e898-4a59-a564-a6a06442457a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:05.331386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f1f85051-e898-4a59-a564-a6a06442457a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:05.389275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:05.411323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882acab1-6f7d-4408-a6a2-15f9235b5933 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:05.412795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882acab1-6f7d-4408-a6a2-15f9235b5933 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:05.423444 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 930/1886] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:05.424907 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a7c09-a886-4bcd-bf15-5522f2680a6c None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:05.425196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a7c09-a886-4bcd-bf15-5522f2680a6c None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:05.425476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a7c09-a886-4bcd-bf15-5522f2680a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:05.432291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f85051-e898-4a59-a564-a6a06442457a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:05.434214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f85051-e898-4a59-a564-a6a06442457a service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:05.447527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 957/1887] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:05.454928 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6f8a7c09-a886-4bcd-bf15-5522f2680a6c None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:05.456650 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6f8a7c09-a886-4bcd-bf15-5522f2680a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:05.469904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 931/1888] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:05.480497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f85051-e898-4a59-a564-a6a06442457a service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:05.482762 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f1f85051-e898-4a59-a564-a6a06442457a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:05.497764 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f85051-e898-4a59-a564-a6a06442457a service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:05.503649 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9fcfc-9560-4ec0-b406-7f4fb7fccbfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:05.504418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9fcfc-9560-4ec0-b406-7f4fb7fccbfe service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:05.505761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9fcfc-9560-4ec0-b406-7f4fb7fccbfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:05.506064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9fcfc-9560-4ec0-b406-7f4fb7fccbfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:05.520285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9fcfc-9560-4ec0-b406-7f4fb7fccbfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:05.522679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9fcfc-9560-4ec0-b406-7f4fb7fccbfe service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:05.549515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 958/1889] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:06.153801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a7c09-a886-4bcd-bf15-5522f2680a6c None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:06.155440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6f8a7c09-a886-4bcd-bf15-5522f2680a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:06.179803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a7c09-a886-4bcd-bf15-5522f2680a6c None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:06.195656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:06.196350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:06.196628 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:06.196900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] PATH_INFO: `/v3/projects/0423cf0e8bd84438a4913be45146c31d` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:06.198065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0423cf0e8bd84438a4913be45146c31d)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:06.200285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:06.281828 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0423cf0e8bd84438a4913be45146c31d', 'request_id': 'req-972080cb-5373-4658-b9d9-ad41f03379c3'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.290420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0423cf0e8bd84438a4913be45146c31d', 'request_id': 'req-972080cb-5373-4658-b9d9-ad41f03379c3'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.296287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0423cf0e8bd84438a4913be45146c31d', 'request_id': 'req-972080cb-5373-4658-b9d9-ad41f03379c3'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.307971 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] The token cache is being invalidate because project 0423cf0e8bd84438a4913be45146c31d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:06.308283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.309415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.310910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 932/1890] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:06 2022] DELETE /identity/v3/projects/0423cf0e8bd84438a4913be45146c31d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:06.316966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bae5ac-d4c0-45ad-a334-e7e79062e003 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:06.318133 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bae5ac-d4c0-45ad-a334-e7e79062e003 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:06.318596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bae5ac-d4c0-45ad-a334-e7e79062e003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:06.342405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-99bae5ac-d4c0-45ad-a334-e7e79062e003 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:06.343721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-99bae5ac-d4c0-45ad-a334-e7e79062e003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:06.383847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:06.386346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:06.422354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 959/1891] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:06.426908 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eaefc7-87cb-4c14-8b39-9e8db3ffbc53 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:06.427509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eaefc7-87cb-4c14-8b39-9e8db3ffbc53 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:06.428969 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eaefc7-87cb-4c14-8b39-9e8db3ffbc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:06.457659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-29eaefc7-87cb-4c14-8b39-9e8db3ffbc53 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:06.458971 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-29eaefc7-87cb-4c14-8b39-9e8db3ffbc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:06.472228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 960/1892] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:06.473582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eaefc7-87cb-4c14-8b39-9e8db3ffbc53 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:06.480930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:06.484960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eaefc7-87cb-4c14-8b39-9e8db3ffbc53 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:06.491990 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6c6a3-d8ae-4fec-8aa0-6869ab78356f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:06.492197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6c6a3-d8ae-4fec-8aa0-6869ab78356f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:06.492379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6c6a3-d8ae-4fec-8aa0-6869ab78356f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:06.492660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6c6a3-d8ae-4fec-8aa0-6869ab78356f admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:06.493312 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6c6a3-d8ae-4fec-8aa0-6869ab78356f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:37:06.493619 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6c6a3-d8ae-4fec-8aa0-6869ab78356f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:06.495713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6c6a3-d8ae-4fec-8aa0-6869ab78356f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:06.500214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972080cb-5373-4658-b9d9-ad41f03379c3 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:06.505095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:06.505557 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:06.505833 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:06.506103 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] PATH_INFO: `/v3/projects/d7d49f035a58472ab3d9318a3b46994d` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:06.507796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7d49f035a58472ab3d9318a3b46994d)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:06.512115 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 961/1893] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:37:06.512264 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:06.513159 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6c6a3-d8ae-4fec-8aa0-6869ab78356f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:06.526547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6c6a3-d8ae-4fec-8aa0-6869ab78356f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:06.549946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e8e07-44b3-4a9b-8582-174ca61c932d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:06.550551 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e8e07-44b3-4a9b-8582-174ca61c932d admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:06.550829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e8e07-44b3-4a9b-8582-174ca61c932d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:06.551099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e8e07-44b3-4a9b-8582-174ca61c932d admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:06.551739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e8e07-44b3-4a9b-8582-174ca61c932d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:06.553839 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e8e07-44b3-4a9b-8582-174ca61c932d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:06.588509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7d49f035a58472ab3d9318a3b46994d', 'request_id': 'req-eee1e30f-2917-4fd7-a695-b178fba0a7a1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.595516 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 962/1894] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:06 2022] POST /identity/v3/projects => generated 375 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:06.600158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e8e07-44b3-4a9b-8582-174ca61c932d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:06.601279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7d49f035a58472ab3d9318a3b46994d', 'request_id': 'req-eee1e30f-2917-4fd7-a695-b178fba0a7a1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.603748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:06.609178 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7d49f035a58472ab3d9318a3b46994d', 'request_id': 'req-eee1e30f-2917-4fd7-a695-b178fba0a7a1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.639655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] The token cache is being invalidate because project d7d49f035a58472ab3d9318a3b46994d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:06.639655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.639655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.639655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 933/1895] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:06 2022] DELETE /identity/v3/projects/d7d49f035a58472ab3d9318a3b46994d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:06.639655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:06.640152 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:06.680295 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e8e07-44b3-4a9b-8582-174ca61c932d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:06.685834 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828b59a-0bae-4f6a-ab3a-ce201a044321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:06.686592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828b59a-0bae-4f6a-ab3a-ce201a044321 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:06.686871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828b59a-0bae-4f6a-ab3a-ce201a044321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:06.687139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828b59a-0bae-4f6a-ab3a-ce201a044321 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:06.687772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828b59a-0bae-4f6a-ab3a-ce201a044321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:06.690114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828b59a-0bae-4f6a-ab3a-ce201a044321 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:06.718055 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 963/1896] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:06 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:37:06.720344 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1e30f-2917-4fd7-a695-b178fba0a7a1 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:06.723439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828b59a-0bae-4f6a-ab3a-ce201a044321 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:06.726481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0828b59a-0bae-4f6a-ab3a-ce201a044321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:06.727025 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787e176-1a56-4abb-89bb-bc16a35217b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:06.727683 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787e176-1a56-4abb-89bb-bc16a35217b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:06.727983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787e176-1a56-4abb-89bb-bc16a35217b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:06.728253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787e176-1a56-4abb-89bb-bc16a35217b5 admin admin] PATH_INFO: `/v3/users/a4ac97b743d94c45bcb7af639318b669` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:06.729408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787e176-1a56-4abb-89bb-bc16a35217b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4ac97b743d94c45bcb7af639318b669)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:06.731371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787e176-1a56-4abb-89bb-bc16a35217b5 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:06.742616 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828b59a-0bae-4f6a-ab3a-ce201a044321 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:06.752088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8afbc-dcad-4b09-9ecc-794ed5d022c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:06.752738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8afbc-dcad-4b09-9ecc-794ed5d022c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:06.752999 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8afbc-dcad-4b09-9ecc-794ed5d022c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:06.753266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8afbc-dcad-4b09-9ecc-794ed5d022c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:06.756488 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8afbc-dcad-4b09-9ecc-794ed5d022c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:06.758058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8afbc-dcad-4b09-9ecc-794ed5d022c4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:06.774189 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 964/1897] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:06.777094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8afbc-dcad-4b09-9ecc-794ed5d022c4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:06.787486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8afbc-dcad-4b09-9ecc-794ed5d022c4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:06.792630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42609fa9-247f-4124-8569-a2c3257a44e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:06.793232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42609fa9-247f-4124-8569-a2c3257a44e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:06.793936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42609fa9-247f-4124-8569-a2c3257a44e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:06.794203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42609fa9-247f-4124-8569-a2c3257a44e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:06.794895 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42609fa9-247f-4124-8569-a2c3257a44e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:06.796726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42609fa9-247f-4124-8569-a2c3257a44e5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:06.800629 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f787e176-1a56-4abb-89bb-bc16a35217b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4ac97b743d94c45bcb7af639318b669'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.807924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 965/1898] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:06.812196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42609fa9-247f-4124-8569-a2c3257a44e5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:06.817719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f787e176-1a56-4abb-89bb-bc16a35217b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4ac97b743d94c45bcb7af639318b669'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.821933 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42609fa9-247f-4124-8569-a2c3257a44e5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:06.827309 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90129b7-262f-4ede-91ac-d369ded210cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:06.827605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f787e176-1a56-4abb-89bb-bc16a35217b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4ac97b743d94c45bcb7af639318b669'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.828185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90129b7-262f-4ede-91ac-d369ded210cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:06.828871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90129b7-262f-4ede-91ac-d369ded210cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:06.829246 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90129b7-262f-4ede-91ac-d369ded210cb admin admin] PATH_INFO: `/v3/projects/b6185130f1c64c8da2923854ef7bad1c/users/77cd5fc8621e4d879016ffc6639ab5c6/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:06.837075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f787e176-1a56-4abb-89bb-bc16a35217b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4ac97b743d94c45bcb7af639318b669'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.837737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f787e176-1a56-4abb-89bb-bc16a35217b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4ac97b743d94c45bcb7af639318b669'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:06.845521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 934/1899] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:06 2022] DELETE /identity/v3/users/a4ac97b743d94c45bcb7af639318b669 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:37:06.845718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90129b7-262f-4ede-91ac-d369ded210cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6185130f1c64c8da2923854ef7bad1c, user_id=77cd5fc8621e4d879016ffc6639ab5c6, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:06.848402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90129b7-262f-4ede-91ac-d369ded210cb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:06.864286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 966/1900] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:06 2022] PUT /identity/v3/projects/b6185130f1c64c8da2923854ef7bad1c/users/77cd5fc8621e4d879016ffc6639ab5c6/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:37:06.875833 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c18116-20bb-4159-80da-fcf489b1c656 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:06.876135 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c18116-20bb-4159-80da-fcf489b1c656 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:06.876413 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c18116-20bb-4159-80da-fcf489b1c656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:06.913889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-a5c18116-20bb-4159-80da-fcf489b1c656 None None] MFA Rules not processed for user `77cd5fc8621e4d879016ffc6639ab5c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:06.915134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a5c18116-20bb-4159-80da-fcf489b1c656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:07.033990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:07.053822 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 935/1901] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:37:07.065102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90129b7-262f-4ede-91ac-d369ded210cb admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:07.069696 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d90129b7-262f-4ede-91ac-d369ded210cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:07.176496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90129b7-262f-4ede-91ac-d369ded210cb admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:07.191606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fc1c9-44e4-420e-bdab-329c860ce56d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:07.192262 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fc1c9-44e4-420e-bdab-329c860ce56d service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.192531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fc1c9-44e4-420e-bdab-329c860ce56d service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.192796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fc1c9-44e4-420e-bdab-329c860ce56d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.208595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fc1c9-44e4-420e-bdab-329c860ce56d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:07.210595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fc1c9-44e4-420e-bdab-329c860ce56d service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:07.228806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c18116-20bb-4159-80da-fcf489b1c656 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:07.230374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a5c18116-20bb-4159-80da-fcf489b1c656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:07.235926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 967/1902] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:07.258980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c18116-20bb-4159-80da-fcf489b1c656 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:07.269698 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15e123-e778-468c-9eee-67b52a00d542 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:07.270659 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15e123-e778-468c-9eee-67b52a00d542 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.271326 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15e123-e778-468c-9eee-67b52a00d542 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.272586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15e123-e778-468c-9eee-67b52a00d542 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.277185 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8e15e123-e778-468c-9eee-67b52a00d542 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:07.291328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fc1c9-44e4-420e-bdab-329c860ce56d service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:07.292877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bf8fc1c9-44e4-420e-bdab-329c860ce56d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:07.321503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:07.440237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fc1c9-44e4-420e-bdab-329c860ce56d service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:07.446240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15e123-e778-468c-9eee-67b52a00d542 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:07.447693 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15e123-e778-468c-9eee-67b52a00d542 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:07.454625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bdda2e-608f-4611-99fb-3a8cb054a32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:07.455276 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bdda2e-608f-4611-99fb-3a8cb054a32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.455586 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bdda2e-608f-4611-99fb-3a8cb054a32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.456119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bdda2e-608f-4611-99fb-3a8cb054a32e admin admin] PATH_INFO: `/v3/projects/10a58e047e3a48ff9adf4c84422fd7ef` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.457169 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bdda2e-608f-4611-99fb-3a8cb054a32e admin admin] RBAC: Authorizing `identity:delete_project(project_id=10a58e047e3a48ff9adf4c84422fd7ef)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:07.459135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bdda2e-608f-4611-99fb-3a8cb054a32e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:07.475278 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 936/1903] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:37:07.547328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-19bdda2e-608f-4611-99fb-3a8cb054a32e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10a58e047e3a48ff9adf4c84422fd7ef', 'request_id': 'req-19bdda2e-608f-4611-99fb-3a8cb054a32e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:07.555014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-19bdda2e-608f-4611-99fb-3a8cb054a32e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10a58e047e3a48ff9adf4c84422fd7ef', 'request_id': 'req-19bdda2e-608f-4611-99fb-3a8cb054a32e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:07.564149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-19bdda2e-608f-4611-99fb-3a8cb054a32e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10a58e047e3a48ff9adf4c84422fd7ef', 'request_id': 'req-19bdda2e-608f-4611-99fb-3a8cb054a32e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:07.590878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-19bdda2e-608f-4611-99fb-3a8cb054a32e admin admin] The token cache is being invalidate because project 10a58e047e3a48ff9adf4c84422fd7ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:07.598798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-19bdda2e-608f-4611-99fb-3a8cb054a32e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:07.604841 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-19bdda2e-608f-4611-99fb-3a8cb054a32e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:07.607348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 968/1904] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:07 2022] DELETE /identity/v3/projects/10a58e047e3a48ff9adf4c84422fd7ef => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:37:07.612687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f5fcb-9200-44ae-90f9-410ee51ab140 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.613061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f5fcb-9200-44ae-90f9-410ee51ab140 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.613333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f5fcb-9200-44ae-90f9-410ee51ab140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.638469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-834f5fcb-9200-44ae-90f9-410ee51ab140 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:07.639797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-834f5fcb-9200-44ae-90f9-410ee51ab140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:07.809464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a3824-d3bc-44de-ae8d-57603264689e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.809646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 937/1905] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:07.810124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a3824-d3bc-44de-ae8d-57603264689e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.810578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a3824-d3bc-44de-ae8d-57603264689e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.857277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f65a3824-d3bc-44de-ae8d-57603264689e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:07.858914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f65a3824-d3bc-44de-ae8d-57603264689e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:07.883909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 969/1906] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:37:07.886298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f5fcb-9200-44ae-90f9-410ee51ab140 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:07.900822 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f5fcb-9200-44ae-90f9-410ee51ab140 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:07.918725 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b377028-7298-4b86-be4e-6e5948cea47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:07.919068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b377028-7298-4b86-be4e-6e5948cea47b admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:07.919446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b377028-7298-4b86-be4e-6e5948cea47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:07.919814 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b377028-7298-4b86-be4e-6e5948cea47b admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:07.920618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b377028-7298-4b86-be4e-6e5948cea47b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:37:07.921027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b377028-7298-4b86-be4e-6e5948cea47b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:07.922741 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b377028-7298-4b86-be4e-6e5948cea47b admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:07.947872 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 938/1907] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:37:07.954195 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a3824-d3bc-44de-ae8d-57603264689e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:07.973163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a3824-d3bc-44de-ae8d-57603264689e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:07.999719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3255a59-5e4e-4bd9-aa2e-539266808ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:08.003468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3255a59-5e4e-4bd9-aa2e-539266808ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:08.003815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3255a59-5e4e-4bd9-aa2e-539266808ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:08.004095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3255a59-5e4e-4bd9-aa2e-539266808ee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:08.009917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3255a59-5e4e-4bd9-aa2e-539266808ee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:08.011833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3255a59-5e4e-4bd9-aa2e-539266808ee6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:08.087490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 970/1908] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:07 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:08.092549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b377028-7298-4b86-be4e-6e5948cea47b admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:08.114060 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b377028-7298-4b86-be4e-6e5948cea47b admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:08.202737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a10f38-fae7-446c-81a1-f0752770777d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:08.203354 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a10f38-fae7-446c-81a1-f0752770777d admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:08.203982 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a10f38-fae7-446c-81a1-f0752770777d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:08.204383 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a10f38-fae7-446c-81a1-f0752770777d admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:08.205192 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a10f38-fae7-446c-81a1-f0752770777d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:08.207192 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a10f38-fae7-446c-81a1-f0752770777d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:08.236129 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 939/1909] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:08 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:08.238641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3255a59-5e4e-4bd9-aa2e-539266808ee6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:08.253811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3255a59-5e4e-4bd9-aa2e-539266808ee6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:08.258466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bbd69-7e33-464f-ba19-8c5db6538d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:08.259093 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bbd69-7e33-464f-ba19-8c5db6538d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:08.259369 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bbd69-7e33-464f-ba19-8c5db6538d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:08.259644 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bbd69-7e33-464f-ba19-8c5db6538d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:08.260451 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4bbd69-7e33-464f-ba19-8c5db6538d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:08.262598 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4bbd69-7e33-464f-ba19-8c5db6538d63 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:08.276722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 971/1910] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:08.280970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a10f38-fae7-446c-81a1-f0752770777d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:08.292730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a10f38-fae7-446c-81a1-f0752770777d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:08.298317 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de7f87-bee4-4536-aa9c-3ce32eb7f231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:08.299182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de7f87-bee4-4536-aa9c-3ce32eb7f231 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:08.299438 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de7f87-bee4-4536-aa9c-3ce32eb7f231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:08.299746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de7f87-bee4-4536-aa9c-3ce32eb7f231 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:08.300444 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de7f87-bee4-4536-aa9c-3ce32eb7f231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:08.302059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de7f87-bee4-4536-aa9c-3ce32eb7f231 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:08.310441 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:08.327949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 940/1911] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:37:08.328196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bbd69-7e33-464f-ba19-8c5db6538d63 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:08.336218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bbd69-7e33-464f-ba19-8c5db6538d63 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:08.342729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2bdf9-4176-4966-9bae-97f7d5a9109a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:08.343386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2bdf9-4176-4966-9bae-97f7d5a9109a admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:08.343659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2bdf9-4176-4966-9bae-97f7d5a9109a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:08.343956 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2bdf9-4176-4966-9bae-97f7d5a9109a admin admin] PATH_INFO: `/v3/projects/5355f2d36227475490b7e52073a5dd10/users/77fc885e76804bdc8c837853701dd598/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:08.363363 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2bdf9-4176-4966-9bae-97f7d5a9109a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5355f2d36227475490b7e52073a5dd10, user_id=77fc885e76804bdc8c837853701dd598, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:08.366148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2bdf9-4176-4966-9bae-97f7d5a9109a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:08.397103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 972/1912] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:08 2022] PUT /identity/v3/projects/5355f2d36227475490b7e52073a5dd10/users/77fc885e76804bdc8c837853701dd598/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:37:08.399331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97c7fc-8a66-4956-a6d0-4b011727cf66 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:08.399621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97c7fc-8a66-4956-a6d0-4b011727cf66 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:08.400209 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d97c7fc-8a66-4956-a6d0-4b011727cf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:08.431057 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-3d97c7fc-8a66-4956-a6d0-4b011727cf66 None None] MFA Rules not processed for user `77fc885e76804bdc8c837853701dd598`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:08.432714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3d97c7fc-8a66-4956-a6d0-4b011727cf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:08.438773 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2bdf9-4176-4966-9bae-97f7d5a9109a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:08.441436 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9ac2bdf9-4176-4966-9bae-97f7d5a9109a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:08.510448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:08.531002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2bdf9-4176-4966-9bae-97f7d5a9109a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:08.535523 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0befbc-7a24-4bdc-ae16-87c7df77cd49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:08.536828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0befbc-7a24-4bdc-ae16-87c7df77cd49 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:08.537222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0befbc-7a24-4bdc-ae16-87c7df77cd49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:08.537637 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0befbc-7a24-4bdc-ae16-87c7df77cd49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:08.541024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-aa0befbc-7a24-4bdc-ae16-87c7df77cd49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:08.547304 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 941/1913] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:08.548861 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97c7fc-8a66-4956-a6d0-4b011727cf66 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:08.618442 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0befbc-7a24-4bdc-ae16-87c7df77cd49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:08.618790 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d97c7fc-8a66-4956-a6d0-4b011727cf66 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:08.620574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0befbc-7a24-4bdc-ae16-87c7df77cd49 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:08.624786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fc24d-1ed9-4d61-8c4a-4ef023a4c7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:08.625416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fc24d-1ed9-4d61-8c4a-4ef023a4c7c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:08.625868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fc24d-1ed9-4d61-8c4a-4ef023a4c7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:08.626137 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fc24d-1ed9-4d61-8c4a-4ef023a4c7c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:08.626776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fc24d-1ed9-4d61-8c4a-4ef023a4c7c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:08.629381 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fc24d-1ed9-4d61-8c4a-4ef023a4c7c3 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:08.632247 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 973/1914] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:08.669173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 942/1915] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:08 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:08.674604 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0befbc-7a24-4bdc-ae16-87c7df77cd49 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:08.688254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0befbc-7a24-4bdc-ae16-87c7df77cd49 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:08.794610 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcc656-59be-45ba-8ee4-2fcbf5d97241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:08.795308 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcc656-59be-45ba-8ee4-2fcbf5d97241 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:08.795595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcc656-59be-45ba-8ee4-2fcbf5d97241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:08.795881 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcc656-59be-45ba-8ee4-2fcbf5d97241 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:08.796761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcc656-59be-45ba-8ee4-2fcbf5d97241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:08.799780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcc656-59be-45ba-8ee4-2fcbf5d97241 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:08.829166 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 974/1916] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:08 2022] POST /identity/v3/users => generated 297 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:08.841392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fc24d-1ed9-4d61-8c4a-4ef023a4c7c3 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:08.853702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fc24d-1ed9-4d61-8c4a-4ef023a4c7c3 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:08.859187 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c713d1-1367-4212-a924-d2a337f87839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:08.859806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c713d1-1367-4212-a924-d2a337f87839 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:08.860118 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c713d1-1367-4212-a924-d2a337f87839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:08.860392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c713d1-1367-4212-a924-d2a337f87839 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:08.861447 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c713d1-1367-4212-a924-d2a337f87839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:08.863074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c713d1-1367-4212-a924-d2a337f87839 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:08.897927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 943/1917] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:08.899466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcc656-59be-45ba-8ee4-2fcbf5d97241 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:08.916530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcc656-59be-45ba-8ee4-2fcbf5d97241 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:08.923802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b5693-1995-48f2-a140-f629bac40cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:08.924519 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b5693-1995-48f2-a140-f629bac40cdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:08.924800 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b5693-1995-48f2-a140-f629bac40cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:08.925070 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b5693-1995-48f2-a140-f629bac40cdf admin admin] PATH_INFO: `/v3/projects/c364629aee884a6f8a909f278044bf61/users/28cb549ce9604597b36e4deec755c884/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:08.946510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b5693-1995-48f2-a140-f629bac40cdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c364629aee884a6f8a909f278044bf61, user_id=28cb549ce9604597b36e4deec755c884, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:08.949463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b5693-1995-48f2-a140-f629bac40cdf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:08.967711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 975/1918] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:08 2022] PUT /identity/v3/projects/c364629aee884a6f8a909f278044bf61/users/28cb549ce9604597b36e4deec755c884/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:08.972209 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c713d1-1367-4212-a924-d2a337f87839 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:09.003776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c713d1-1367-4212-a924-d2a337f87839 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:09.091938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a6d57-c0df-47f0-9f91-cf29a30567ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:09.093168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a6d57-c0df-47f0-9f91-cf29a30567ed admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:09.093648 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a6d57-c0df-47f0-9f91-cf29a30567ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:09.094154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a6d57-c0df-47f0-9f91-cf29a30567ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:09.095237 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a6d57-c0df-47f0-9f91-cf29a30567ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:09.097144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a6d57-c0df-47f0-9f91-cf29a30567ed admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:09.105246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:09.112665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 944/1919] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:09.118172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b5693-1995-48f2-a140-f629bac40cdf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:09.136708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b5693-1995-48f2-a140-f629bac40cdf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:09.146885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ff65f-0d83-4ffb-982c-e4ed9771ce14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:09.147313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ff65f-0d83-4ffb-982c-e4ed9771ce14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:09.147608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ff65f-0d83-4ffb-982c-e4ed9771ce14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:09.147882 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ff65f-0d83-4ffb-982c-e4ed9771ce14 admin admin] PATH_INFO: `/v3/projects/c364629aee884a6f8a909f278044bf61/users/28cb549ce9604597b36e4deec755c884/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:09.150129 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ff65f-0d83-4ffb-982c-e4ed9771ce14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c364629aee884a6f8a909f278044bf61, user_id=28cb549ce9604597b36e4deec755c884, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:09.153093 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ff65f-0d83-4ffb-982c-e4ed9771ce14 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:09.164901 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 976/1920] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:09 2022] PUT /identity/v3/projects/c364629aee884a6f8a909f278044bf61/users/28cb549ce9604597b36e4deec755c884/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:09.177620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0f85a-649f-4eac-b166-c5a9dce4a414 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:09.178071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0f85a-649f-4eac-b166-c5a9dce4a414 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:09.178421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0f85a-649f-4eac-b166-c5a9dce4a414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:09.216780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-25d0f85a-649f-4eac-b166-c5a9dce4a414 None None] MFA Rules not processed for user `28cb549ce9604597b36e4deec755c884`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:09.218742 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-25d0f85a-649f-4eac-b166-c5a9dce4a414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:09.333913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 945/1921] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:09.346664 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ff65f-0d83-4ffb-982c-e4ed9771ce14 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:09.430028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:09.442002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ff65f-0d83-4ffb-982c-e4ed9771ce14 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:09.445924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18786b-578a-488f-919c-5f895b793da7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:09.446778 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18786b-578a-488f-919c-5f895b793da7 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:09.447056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18786b-578a-488f-919c-5f895b793da7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:09.447343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18786b-578a-488f-919c-5f895b793da7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:09.459510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18786b-578a-488f-919c-5f895b793da7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:09.459868 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18786b-578a-488f-919c-5f895b793da7 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:09.472441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 977/1922] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:10.053906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0f85a-649f-4eac-b166-c5a9dce4a414 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:10.166918 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0f85a-649f-4eac-b166-c5a9dce4a414 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:10.181656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23976c4b-3ed7-4e55-84bc-d39bb724ac37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:10.182813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23976c4b-3ed7-4e55-84bc-d39bb724ac37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:10.183637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23976c4b-3ed7-4e55-84bc-d39bb724ac37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:10.184593 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23976c4b-3ed7-4e55-84bc-d39bb724ac37 admin admin] PATH_INFO: `/v3/users/77fc885e76804bdc8c837853701dd598` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:10.187212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23976c4b-3ed7-4e55-84bc-d39bb724ac37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77fc885e76804bdc8c837853701dd598)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:10.190743 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23976c4b-3ed7-4e55-84bc-d39bb724ac37 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:10.289768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-23976c4b-3ed7-4e55-84bc-d39bb724ac37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77fc885e76804bdc8c837853701dd598'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:10.308272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-23976c4b-3ed7-4e55-84bc-d39bb724ac37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77fc885e76804bdc8c837853701dd598'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:10.324996 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-23976c4b-3ed7-4e55-84bc-d39bb724ac37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77fc885e76804bdc8c837853701dd598'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:10.340627 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-23976c4b-3ed7-4e55-84bc-d39bb724ac37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77fc885e76804bdc8c837853701dd598'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:10.342667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-23976c4b-3ed7-4e55-84bc-d39bb724ac37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77fc885e76804bdc8c837853701dd598'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:10.353374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 946/1923] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:10 2022] DELETE /identity/v3/users/77fc885e76804bdc8c837853701dd598 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:10.358944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212620e7-ef5c-4375-b9fb-b8330dcefc84 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:10.359521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212620e7-ef5c-4375-b9fb-b8330dcefc84 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:10.359602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212620e7-ef5c-4375-b9fb-b8330dcefc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:10.397629 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-212620e7-ef5c-4375-b9fb-b8330dcefc84 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:10.399713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-212620e7-ef5c-4375-b9fb-b8330dcefc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:10.604849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 978/1924] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:10.619699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23976c4b-3ed7-4e55-84bc-d39bb724ac37 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:10.624536 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-23976c4b-3ed7-4e55-84bc-d39bb724ac37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:10.641425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:10.749612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23976c4b-3ed7-4e55-84bc-d39bb724ac37 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:10.768074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706fd5e-9729-4d51-a394-816fbfd7123a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:10.768737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706fd5e-9729-4d51-a394-816fbfd7123a service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:10.769016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706fd5e-9729-4d51-a394-816fbfd7123a service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:10.769290 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706fd5e-9729-4d51-a394-816fbfd7123a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:10.802567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706fd5e-9729-4d51-a394-816fbfd7123a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:10.805346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706fd5e-9729-4d51-a394-816fbfd7123a service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:10.829437 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 947/1925] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:37:11.791130 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212620e7-ef5c-4375-b9fb-b8330dcefc84 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:11.793184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-212620e7-ef5c-4375-b9fb-b8330dcefc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:11.819754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212620e7-ef5c-4375-b9fb-b8330dcefc84 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:11.842866 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:11.843964 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:11.855282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:11.855745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] PATH_INFO: `/v3/projects/5355f2d36227475490b7e52073a5dd10` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:11.857877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5355f2d36227475490b7e52073a5dd10)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:11.864301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:12.004122 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5355f2d36227475490b7e52073a5dd10', 'request_id': 'req-5e19aeee-f657-41f4-80c0-055582ddc674'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.015434 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5355f2d36227475490b7e52073a5dd10', 'request_id': 'req-5e19aeee-f657-41f4-80c0-055582ddc674'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.028442 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5355f2d36227475490b7e52073a5dd10', 'request_id': 'req-5e19aeee-f657-41f4-80c0-055582ddc674'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.065357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] The token cache is being invalidate because project 5355f2d36227475490b7e52073a5dd10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:12.065779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.066320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.068463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 979/1926] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:11 2022] DELETE /identity/v3/projects/5355f2d36227475490b7e52073a5dd10 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:12.072639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706fd5e-9729-4d51-a394-816fbfd7123a service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:12.084816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706fd5e-9729-4d51-a394-816fbfd7123a service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:12.185421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e4aa2-fc68-4692-b7af-7fca0d03a011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:12.186250 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e4aa2-fc68-4692-b7af-7fca0d03a011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:12.186517 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e4aa2-fc68-4692-b7af-7fca0d03a011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:12.187689 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e4aa2-fc68-4692-b7af-7fca0d03a011 admin admin] PATH_INFO: `/v3/users/28cb549ce9604597b36e4deec755c884` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:12.189093 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e4aa2-fc68-4692-b7af-7fca0d03a011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28cb549ce9604597b36e4deec755c884)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:12.191444 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e4aa2-fc68-4692-b7af-7fca0d03a011 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:12.263352 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3b2e4aa2-fc68-4692-b7af-7fca0d03a011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28cb549ce9604597b36e4deec755c884'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.280757 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3b2e4aa2-fc68-4692-b7af-7fca0d03a011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28cb549ce9604597b36e4deec755c884'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.291746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3b2e4aa2-fc68-4692-b7af-7fca0d03a011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28cb549ce9604597b36e4deec755c884'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.304945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3b2e4aa2-fc68-4692-b7af-7fca0d03a011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28cb549ce9604597b36e4deec755c884'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.305748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3b2e4aa2-fc68-4692-b7af-7fca0d03a011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28cb549ce9604597b36e4deec755c884'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.314574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 948/1927] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:12 2022] DELETE /identity/v3/users/28cb549ce9604597b36e4deec755c884 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:12.686903 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:12.689298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:12.714240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19aeee-f657-41f4-80c0-055582ddc674 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:12.792779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:12.793457 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:12.794075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:12.794346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] PATH_INFO: `/v3/projects/c364629aee884a6f8a909f278044bf61` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:12.796258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c364629aee884a6f8a909f278044bf61)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:12.798169 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:12.810907 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:12.868786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c364629aee884a6f8a909f278044bf61', 'request_id': 'req-7a43ab51-3363-495f-a739-91b86763c207'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.876288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c364629aee884a6f8a909f278044bf61', 'request_id': 'req-7a43ab51-3363-495f-a739-91b86763c207'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.883688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c364629aee884a6f8a909f278044bf61', 'request_id': 'req-7a43ab51-3363-495f-a739-91b86763c207'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.915242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] The token cache is being invalidate because project c364629aee884a6f8a909f278044bf61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:12.915621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.916384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:12.919426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 980/1928] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:12 2022] DELETE /identity/v3/projects/c364629aee884a6f8a909f278044bf61 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:12.926402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952cb33-d748-4056-a354-548b00e30d18 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:12.926679 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952cb33-d748-4056-a354-548b00e30d18 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:12.926943 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952cb33-d748-4056-a354-548b00e30d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:12.990993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-9952cb33-d748-4056-a354-548b00e30d18 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:12.992958 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9952cb33-d748-4056-a354-548b00e30d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:13.040509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:13.130814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:13.132595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:13.148107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 949/1929] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:13.152833 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0b98b-651b-4403-ad48-1da5e8434148 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.153271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0b98b-651b-4403-ad48-1da5e8434148 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.154473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0b98b-651b-4403-ad48-1da5e8434148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.154605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43ab51-3363-495f-a739-91b86763c207 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:13.177191 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84427b71-64ad-4157-a8df-2673977914cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:13.177905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84427b71-64ad-4157-a8df-2673977914cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.178214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84427b71-64ad-4157-a8df-2673977914cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.178528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84427b71-64ad-4157-a8df-2673977914cf admin admin] PATH_INFO: `/v3/users/63a45eaebf584b62b1a53336e3728060` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.179865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84427b71-64ad-4157-a8df-2673977914cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a45eaebf584b62b1a53336e3728060)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:13.181975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84427b71-64ad-4157-a8df-2673977914cf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:13.201901 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-dbe0b98b-651b-4403-ad48-1da5e8434148 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:13.207874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-dbe0b98b-651b-4403-ad48-1da5e8434148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:13.227768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 950/1930] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:13.232666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0b98b-651b-4403-ad48-1da5e8434148 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:13.246065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0b98b-651b-4403-ad48-1da5e8434148 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:13.252736 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e8fbf-1c4a-482a-b11c-b06cc07df4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:13.254191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e8fbf-1c4a-482a-b11c-b06cc07df4ef admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.254746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e8fbf-1c4a-482a-b11c-b06cc07df4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.255032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e8fbf-1c4a-482a-b11c-b06cc07df4ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.256091 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e8fbf-1c4a-482a-b11c-b06cc07df4ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:37:13.256478 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e8fbf-1c4a-482a-b11c-b06cc07df4ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:13.259226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e8fbf-1c4a-482a-b11c-b06cc07df4ef admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:13.279127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-84427b71-64ad-4157-a8df-2673977914cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a45eaebf584b62b1a53336e3728060'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:13.283921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 951/1931] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:37:13.284963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e8fbf-1c4a-482a-b11c-b06cc07df4ef admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:13.295629 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-84427b71-64ad-4157-a8df-2673977914cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a45eaebf584b62b1a53336e3728060'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:13.296437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-84427b71-64ad-4157-a8df-2673977914cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a45eaebf584b62b1a53336e3728060'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:13.299299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e8fbf-1c4a-482a-b11c-b06cc07df4ef admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:13.305142 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d7700-6aca-489f-a52d-dd8168a985cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:13.305811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d7700-6aca-489f-a52d-dd8168a985cc admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.306325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d7700-6aca-489f-a52d-dd8168a985cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.306998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d7700-6aca-489f-a52d-dd8168a985cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.308107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d7700-6aca-489f-a52d-dd8168a985cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:13.310392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d7700-6aca-489f-a52d-dd8168a985cc admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:13.330979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-84427b71-64ad-4157-a8df-2673977914cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a45eaebf584b62b1a53336e3728060'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:13.341498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-84427b71-64ad-4157-a8df-2673977914cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a45eaebf584b62b1a53336e3728060'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:13.356387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 952/1932] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:13 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:13.360611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d7700-6aca-489f-a52d-dd8168a985cc admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:13.363477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 981/1933] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:13 2022] DELETE /identity/v3/users/63a45eaebf584b62b1a53336e3728060 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:37:13.366321 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651407c-104b-4201-92b7-89c8a168b103 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.366858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651407c-104b-4201-92b7-89c8a168b103 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.367286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651407c-104b-4201-92b7-89c8a168b103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.367527 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-019d7700-6aca-489f-a52d-dd8168a985cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:13.389526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9651407c-104b-4201-92b7-89c8a168b103 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:13.391260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9651407c-104b-4201-92b7-89c8a168b103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:13.483264 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d7700-6aca-489f-a52d-dd8168a985cc admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:13.533559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 982/1934] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:13.540692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651407c-104b-4201-92b7-89c8a168b103 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:13.544876 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9651407c-104b-4201-92b7-89c8a168b103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:13.563519 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:13.624693 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a240c-7f39-457c-8543-205a2d09192b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:13.627140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a240c-7f39-457c-8543-205a2d09192b admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.628175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a240c-7f39-457c-8543-205a2d09192b admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.628788 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a240c-7f39-457c-8543-205a2d09192b admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.630016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a240c-7f39-457c-8543-205a2d09192b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:13.633341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a240c-7f39-457c-8543-205a2d09192b admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:13.666063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651407c-104b-4201-92b7-89c8a168b103 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:13.676340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 953/1935] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:13 2022] POST /identity/v3/users => generated 293 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:13.680167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbd9a1-c325-4769-b078-818bfcc72654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:13.681155 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbd9a1-c325-4769-b078-818bfcc72654 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.681496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbd9a1-c325-4769-b078-818bfcc72654 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.681866 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a240c-7f39-457c-8543-205a2d09192b admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:13.681997 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbd9a1-c325-4769-b078-818bfcc72654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.698529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbd9a1-c325-4769-b078-818bfcc72654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:13.700344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbd9a1-c325-4769-b078-818bfcc72654 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:13.710806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a240c-7f39-457c-8543-205a2d09192b admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:13.712334 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 983/1936] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:13.728417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a091ab-04bd-44e2-973d-02a166424d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:13.728957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a091ab-04bd-44e2-973d-02a166424d06 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.729278 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a091ab-04bd-44e2-973d-02a166424d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.729563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a091ab-04bd-44e2-973d-02a166424d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.730834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a091ab-04bd-44e2-973d-02a166424d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:13.740644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a091ab-04bd-44e2-973d-02a166424d06 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:13.783949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 954/1937] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:37:13.786930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbd9a1-c325-4769-b078-818bfcc72654 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:13.802521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbd9a1-c325-4769-b078-818bfcc72654 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:13.810302 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4ee28-5112-435c-a76d-03554188b43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:13.811932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4ee28-5112-435c-a76d-03554188b43b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.812481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4ee28-5112-435c-a76d-03554188b43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.813285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4ee28-5112-435c-a76d-03554188b43b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.814250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4ee28-5112-435c-a76d-03554188b43b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:13.816887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4ee28-5112-435c-a76d-03554188b43b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:13.830883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 984/1938] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:13.836931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a091ab-04bd-44e2-973d-02a166424d06 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:13.851006 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a091ab-04bd-44e2-973d-02a166424d06 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:13.858086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee47bb-0349-4668-89c4-1f888f3b7eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:13.858974 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee47bb-0349-4668-89c4-1f888f3b7eea admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.859357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee47bb-0349-4668-89c4-1f888f3b7eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.859722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee47bb-0349-4668-89c4-1f888f3b7eea admin admin] PATH_INFO: `/v3/projects/b27d950768744621a54402eb5fdc247c/users/564e343f84cf4ab89381bb2a79e92a8e/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.881340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:13.910584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee47bb-0349-4668-89c4-1f888f3b7eea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27d950768744621a54402eb5fdc247c, user_id=564e343f84cf4ab89381bb2a79e92a8e, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:13.914349 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee47bb-0349-4668-89c4-1f888f3b7eea admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:13.939933 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 955/1939] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:13 2022] PUT /identity/v3/projects/b27d950768744621a54402eb5fdc247c/users/564e343f84cf4ab89381bb2a79e92a8e/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:13.954412 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d552a6-e58b-4d85-87e1-ee83a563ee98 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:13.956065 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d552a6-e58b-4d85-87e1-ee83a563ee98 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:13.956642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d552a6-e58b-4d85-87e1-ee83a563ee98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:13.990717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d6d552a6-e58b-4d85-87e1-ee83a563ee98 None None] MFA Rules not processed for user `564e343f84cf4ab89381bb2a79e92a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:13.992740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d6d552a6-e58b-4d85-87e1-ee83a563ee98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:14.086715 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee47bb-0349-4668-89c4-1f888f3b7eea admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:14.092438 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 985/1940] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:14.093079 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-41ee47bb-0349-4668-89c4-1f888f3b7eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:14.093287 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d552a6-e58b-4d85-87e1-ee83a563ee98 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:14.240620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d552a6-e58b-4d85-87e1-ee83a563ee98 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:14.256298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419046a6-24aa-45c0-b99e-ff687ab767d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:14.259458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419046a6-24aa-45c0-b99e-ff687ab767d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:14.259458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419046a6-24aa-45c0-b99e-ff687ab767d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:14.259458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419046a6-24aa-45c0-b99e-ff687ab767d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:14.261430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419046a6-24aa-45c0-b99e-ff687ab767d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:14.266820 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419046a6-24aa-45c0-b99e-ff687ab767d3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:14.268457 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee47bb-0349-4668-89c4-1f888f3b7eea admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:14.280091 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:14.282944 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:14.283238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:14.283502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] PATH_INFO: `/v3/projects/5662a0f1df7943e69e506bdb99da4719` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:14.287684 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5662a0f1df7943e69e506bdb99da4719)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:14.289743 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:14.352010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 986/1941] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:14 2022] POST /identity/v3/projects => generated 385 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:14.353344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419046a6-24aa-45c0-b99e-ff687ab767d3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:14.385314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:14.386315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5662a0f1df7943e69e506bdb99da4719', 'request_id': 'req-c5ac7453-bee5-4005-a064-96f283064ad1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:14.400006 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5662a0f1df7943e69e506bdb99da4719', 'request_id': 'req-c5ac7453-bee5-4005-a064-96f283064ad1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:14.420928 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5662a0f1df7943e69e506bdb99da4719', 'request_id': 'req-c5ac7453-bee5-4005-a064-96f283064ad1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:14.455351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] The token cache is being invalidate because project 5662a0f1df7943e69e506bdb99da4719 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:14.455351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:14.455351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:14.455351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 956/1942] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:14 2022] DELETE /identity/v3/projects/5662a0f1df7943e69e506bdb99da4719 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:37:14.457433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:14.458926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:14.494328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419046a6-24aa-45c0-b99e-ff687ab767d3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:14.598199 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac7453-bee5-4005-a064-96f283064ad1 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:14.607767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb37d4-24c0-47d0-a124-2a081806e102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:14.608637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb37d4-24c0-47d0-a124-2a081806e102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:14.608973 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb37d4-24c0-47d0-a124-2a081806e102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:14.609305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb37d4-24c0-47d0-a124-2a081806e102 admin admin] PATH_INFO: `/v3/users/9cf64ba5ef3d475a954846a863ae04f9` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:14.611219 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb37d4-24c0-47d0-a124-2a081806e102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cf64ba5ef3d475a954846a863ae04f9)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:14.613845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb37d4-24c0-47d0-a124-2a081806e102 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:14.632708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26054c2-7125-4d0d-a18e-2847ded7d79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:14.633413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26054c2-7125-4d0d-a18e-2847ded7d79d admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:14.634171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26054c2-7125-4d0d-a18e-2847ded7d79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:14.634454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26054c2-7125-4d0d-a18e-2847ded7d79d admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:14.637687 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26054c2-7125-4d0d-a18e-2847ded7d79d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:14.639064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26054c2-7125-4d0d-a18e-2847ded7d79d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:14.672541 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:14.689417 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 987/1943] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:14 2022] POST /identity/v3/users => generated 293 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:14.693537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26054c2-7125-4d0d-a18e-2847ded7d79d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:14.695227 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e26054c2-7125-4d0d-a18e-2847ded7d79d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:14.710340 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26054c2-7125-4d0d-a18e-2847ded7d79d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:14.716271 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3c56f-2d19-4bb2-b4a0-d466ef48dc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:14.716964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-aafb37d4-24c0-47d0-a124-2a081806e102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cf64ba5ef3d475a954846a863ae04f9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:14.717240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3c56f-2d19-4bb2-b4a0-d466ef48dc67 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:14.717588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3c56f-2d19-4bb2-b4a0-d466ef48dc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:14.717978 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3c56f-2d19-4bb2-b4a0-d466ef48dc67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:14.718693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3c56f-2d19-4bb2-b4a0-d466ef48dc67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:14.720809 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3c56f-2d19-4bb2-b4a0-d466ef48dc67 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:14.735565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 988/1944] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:14.736182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-aafb37d4-24c0-47d0-a124-2a081806e102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cf64ba5ef3d475a954846a863ae04f9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:14.739047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3c56f-2d19-4bb2-b4a0-d466ef48dc67 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:14.744126 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-aafb37d4-24c0-47d0-a124-2a081806e102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cf64ba5ef3d475a954846a863ae04f9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:14.755681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3c56f-2d19-4bb2-b4a0-d466ef48dc67 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:14.760588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-aafb37d4-24c0-47d0-a124-2a081806e102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cf64ba5ef3d475a954846a863ae04f9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:14.772351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-aafb37d4-24c0-47d0-a124-2a081806e102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cf64ba5ef3d475a954846a863ae04f9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:14.776496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3180d-e96e-408f-b832-0bf3f016fa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:14.777175 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3180d-e96e-408f-b832-0bf3f016fa77 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:14.777530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3180d-e96e-408f-b832-0bf3f016fa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:14.777863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3180d-e96e-408f-b832-0bf3f016fa77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:14.778768 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3180d-e96e-408f-b832-0bf3f016fa77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:14.780799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3180d-e96e-408f-b832-0bf3f016fa77 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:14.784177 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 957/1945] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:14 2022] DELETE /identity/v3/users/9cf64ba5ef3d475a954846a863ae04f9 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:37:14.794218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 989/1946] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:37:14.798582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb37d4-24c0-47d0-a124-2a081806e102 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:14.802461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-aafb37d4-24c0-47d0-a124-2a081806e102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:14.894375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb37d4-24c0-47d0-a124-2a081806e102 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:14.905773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9153c-17d3-4c79-883a-fc436de0bade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:14.908378 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9153c-17d3-4c79-883a-fc436de0bade admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:14.909905 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9153c-17d3-4c79-883a-fc436de0bade admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:14.910781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9153c-17d3-4c79-883a-fc436de0bade admin admin] PATH_INFO: `/v3/projects/24ff013d096b4b96aee724255343b972/users/3c66c46d8d8c4e268e1c01ff24ead685/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:14.951696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9153c-17d3-4c79-883a-fc436de0bade admin admin] RBAC: Authorizing `identity:create_grant(project_id=24ff013d096b4b96aee724255343b972, user_id=3c66c46d8d8c4e268e1c01ff24ead685, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:14.960897 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9153c-17d3-4c79-883a-fc436de0bade admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:14.988551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 958/1947] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:14 2022] PUT /identity/v3/projects/24ff013d096b4b96aee724255343b972/users/3c66c46d8d8c4e268e1c01ff24ead685/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:15.000849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863a73d-dc83-48d6-8adf-64c3e8551ea8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:15.001381 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863a73d-dc83-48d6-8adf-64c3e8551ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:15.001381 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863a73d-dc83-48d6-8adf-64c3e8551ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:15.087367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5863a73d-dc83-48d6-8adf-64c3e8551ea8 None None] MFA Rules not processed for user `3c66c46d8d8c4e268e1c01ff24ead685`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:15.089997 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5863a73d-dc83-48d6-8adf-64c3e8551ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:15.198660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 990/1948] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:37:15.199859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9153c-17d3-4c79-883a-fc436de0bade admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:15.292828 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9153c-17d3-4c79-883a-fc436de0bade admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:15.297679 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15c4ad-4788-47ae-ade1-734b0a1a2cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:15.298378 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15c4ad-4788-47ae-ade1-734b0a1a2cca admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:15.298659 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15c4ad-4788-47ae-ade1-734b0a1a2cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:15.298931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15c4ad-4788-47ae-ade1-734b0a1a2cca admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:15.299607 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15c4ad-4788-47ae-ade1-734b0a1a2cca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:15.301846 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15c4ad-4788-47ae-ade1-734b0a1a2cca admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:15.355745 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 959/1949] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:15 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:15.360290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863a73d-dc83-48d6-8adf-64c3e8551ea8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:15.383647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863a73d-dc83-48d6-8adf-64c3e8551ea8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:15.411706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:15.413926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15c4ad-4788-47ae-ade1-734b0a1a2cca admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:15.415771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-fa15c4ad-4788-47ae-ade1-734b0a1a2cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:15.426003 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:15.546040 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c24b3-353f-4d4e-8773-abc3418f0585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:15.546732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c24b3-353f-4d4e-8773-abc3418f0585 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:15.547020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c24b3-353f-4d4e-8773-abc3418f0585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:15.547299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c24b3-353f-4d4e-8773-abc3418f0585 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:15.549519 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c24b3-353f-4d4e-8773-abc3418f0585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:15.550664 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c24b3-353f-4d4e-8773-abc3418f0585 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:15.583465 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 991/1950] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:15 2022] POST /identity/v3/users => generated 298 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:37:15.584583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c24b3-353f-4d4e-8773-abc3418f0585 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:15.598639 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c24b3-353f-4d4e-8773-abc3418f0585 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:15.606848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999d2fb-1d91-48f1-b661-86f8e1e33318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:15.607736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999d2fb-1d91-48f1-b661-86f8e1e33318 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:15.608425 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999d2fb-1d91-48f1-b661-86f8e1e33318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:15.608721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999d2fb-1d91-48f1-b661-86f8e1e33318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:15.609682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999d2fb-1d91-48f1-b661-86f8e1e33318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:15.612411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999d2fb-1d91-48f1-b661-86f8e1e33318 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:15.613469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15c4ad-4788-47ae-ade1-734b0a1a2cca admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:15.638947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 992/1951] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:37:15.639160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:15.639160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:15.639160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:15.639160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] PATH_INFO: `/v3/projects/6619e7c091814b8d8df313991bf2bfda` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:15.640153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999d2fb-1d91-48f1-b661-86f8e1e33318 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:15.641537 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6619e7c091814b8d8df313991bf2bfda)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:15.643720 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:15.666264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999d2fb-1d91-48f1-b661-86f8e1e33318 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:15.700518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8f594-f718-4efd-bff5-189a42340d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:15.701411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8f594-f718-4efd-bff5-189a42340d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:15.701696 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8f594-f718-4efd-bff5-189a42340d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:15.702011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8f594-f718-4efd-bff5-189a42340d48 admin admin] PATH_INFO: `/v3/projects/271d36fe86964d77b0e8c6d7f1caff61/users/3741d039853d418d9bd12760255d66e4/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:15.728070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6619e7c091814b8d8df313991bf2bfda', 'request_id': 'req-ec20ae20-07ce-4833-a434-7c2397ad58c0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:15.744339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6619e7c091814b8d8df313991bf2bfda', 'request_id': 'req-ec20ae20-07ce-4833-a434-7c2397ad58c0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:15.749228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8f594-f718-4efd-bff5-189a42340d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=271d36fe86964d77b0e8c6d7f1caff61, user_id=3741d039853d418d9bd12760255d66e4, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:15.761402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8f594-f718-4efd-bff5-189a42340d48 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:15.770797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6619e7c091814b8d8df313991bf2bfda', 'request_id': 'req-ec20ae20-07ce-4833-a434-7c2397ad58c0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:15.794916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 993/1952] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:15 2022] PUT /identity/v3/projects/271d36fe86964d77b0e8c6d7f1caff61/users/3741d039853d418d9bd12760255d66e4/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:15.798631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8f594-f718-4efd-bff5-189a42340d48 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:15.806644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] The token cache is being invalidate because project 6619e7c091814b8d8df313991bf2bfda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:15.806990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:15.810870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:15.816881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 960/1953] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:15 2022] DELETE /identity/v3/projects/6619e7c091814b8d8df313991bf2bfda => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:37:15.817896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:15.819397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:15.949327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8f594-f718-4efd-bff5-189a42340d48 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:15.964873 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1d9dc-214a-4af0-8af6-54714b13902c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:15.965683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1d9dc-214a-4af0-8af6-54714b13902c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:15.965950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1d9dc-214a-4af0-8af6-54714b13902c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:15.966216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1d9dc-214a-4af0-8af6-54714b13902c admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:15.966862 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1d9dc-214a-4af0-8af6-54714b13902c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:15.970879 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1d9dc-214a-4af0-8af6-54714b13902c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:15.977241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20ae20-07ce-4833-a434-7c2397ad58c0 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:15.997967 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aa86f-18a5-4b1c-8bf7-a7655a5d8ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:16.014386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aa86f-18a5-4b1c-8bf7-a7655a5d8ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:16.014757 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aa86f-18a5-4b1c-8bf7-a7655a5d8ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:16.015026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aa86f-18a5-4b1c-8bf7-a7655a5d8ce3 admin admin] PATH_INFO: `/v3/users/c1d10339f4ef4f5b8b415866adcc45e2` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:16.016712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180aa86f-18a5-4b1c-8bf7-a7655a5d8ce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1d10339f4ef4f5b8b415866adcc45e2)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:16.018606 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180aa86f-18a5-4b1c-8bf7-a7655a5d8ce3 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:16.022976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 994/1954] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:16.022976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1d9dc-214a-4af0-8af6-54714b13902c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:16.028509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b9a1d9dc-214a-4af0-8af6-54714b13902c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:16.063640 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1d9dc-214a-4af0-8af6-54714b13902c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:16.074571 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ff82f-97f2-4301-a8bc-ab09f9ebb4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:16.077016 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ff82f-97f2-4301-a8bc-ab09f9ebb4d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:16.077543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ff82f-97f2-4301-a8bc-ab09f9ebb4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:16.080368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ff82f-97f2-4301-a8bc-ab09f9ebb4d5 admin admin] PATH_INFO: `/v3/projects/271d36fe86964d77b0e8c6d7f1caff61/users/3741d039853d418d9bd12760255d66e4/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:16.083273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ff82f-97f2-4301-a8bc-ab09f9ebb4d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=271d36fe86964d77b0e8c6d7f1caff61, user_id=3741d039853d418d9bd12760255d66e4, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:16.086326 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ff82f-97f2-4301-a8bc-ab09f9ebb4d5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:16.107834 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 995/1955] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:16 2022] PUT /identity/v3/projects/271d36fe86964d77b0e8c6d7f1caff61/users/3741d039853d418d9bd12760255d66e4/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:16.119051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfcdae-e7ca-4e88-bab0-d004dd4d60a3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:16.119339 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfcdae-e7ca-4e88-bab0-d004dd4d60a3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:16.119668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfcdae-e7ca-4e88-bab0-d004dd4d60a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:16.123966 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:16.144321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-180aa86f-18a5-4b1c-8bf7-a7655a5d8ce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1d10339f4ef4f5b8b415866adcc45e2'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.173823 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-180aa86f-18a5-4b1c-8bf7-a7655a5d8ce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1d10339f4ef4f5b8b415866adcc45e2'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.178574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1adfcdae-e7ca-4e88-bab0-d004dd4d60a3 None None] MFA Rules not processed for user `3741d039853d418d9bd12760255d66e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:16.181014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1adfcdae-e7ca-4e88-bab0-d004dd4d60a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:16.190379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-180aa86f-18a5-4b1c-8bf7-a7655a5d8ce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1d10339f4ef4f5b8b415866adcc45e2'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.205933 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-180aa86f-18a5-4b1c-8bf7-a7655a5d8ce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1d10339f4ef4f5b8b415866adcc45e2'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.206663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-180aa86f-18a5-4b1c-8bf7-a7655a5d8ce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1d10339f4ef4f5b8b415866adcc45e2'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:16.220435 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 961/1956] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:15 2022] DELETE /identity/v3/users/c1d10339f4ef4f5b8b415866adcc45e2 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:16.272387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:16.303994 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 996/1957] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:16.312812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aa86f-18a5-4b1c-8bf7-a7655a5d8ce3 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:16.314853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-180aa86f-18a5-4b1c-8bf7-a7655a5d8ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:16.418204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aa86f-18a5-4b1c-8bf7-a7655a5d8ce3 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:16.422622 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc0704-1863-41f7-86c6-2ef2daa74cc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:16.423373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc0704-1863-41f7-86c6-2ef2daa74cc7 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:16.423720 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc0704-1863-41f7-86c6-2ef2daa74cc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:16.424091 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc0704-1863-41f7-86c6-2ef2daa74cc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:16.426532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a2dc0704-1863-41f7-86c6-2ef2daa74cc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:16.498710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc0704-1863-41f7-86c6-2ef2daa74cc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:16.500445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc0704-1863-41f7-86c6-2ef2daa74cc7 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:16.513229 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 962/1958] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:17.089031 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfcdae-e7ca-4e88-bab0-d004dd4d60a3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:17.091537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1adfcdae-e7ca-4e88-bab0-d004dd4d60a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:17.196928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfcdae-e7ca-4e88-bab0-d004dd4d60a3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:17.275696 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c5650-6128-434d-9be9-84d9833cee7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:17.276457 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c5650-6128-434d-9be9-84d9833cee7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:17.276753 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c5650-6128-434d-9be9-84d9833cee7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:17.277027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c5650-6128-434d-9be9-84d9833cee7c admin admin] PATH_INFO: `/v3/projects/d29ce60144344f6e8b24826f0ed0b4bd` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:17.278542 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c5650-6128-434d-9be9-84d9833cee7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d29ce60144344f6e8b24826f0ed0b4bd)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:17.280750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c5650-6128-434d-9be9-84d9833cee7c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:17.358747 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6a8c5650-6128-434d-9be9-84d9833cee7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd29ce60144344f6e8b24826f0ed0b4bd', 'request_id': 'req-6a8c5650-6128-434d-9be9-84d9833cee7c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:17.369464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6a8c5650-6128-434d-9be9-84d9833cee7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd29ce60144344f6e8b24826f0ed0b4bd', 'request_id': 'req-6a8c5650-6128-434d-9be9-84d9833cee7c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:17.377887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6a8c5650-6128-434d-9be9-84d9833cee7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd29ce60144344f6e8b24826f0ed0b4bd', 'request_id': 'req-6a8c5650-6128-434d-9be9-84d9833cee7c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:17.416460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6a8c5650-6128-434d-9be9-84d9833cee7c admin admin] The token cache is being invalidate because project d29ce60144344f6e8b24826f0ed0b4bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:17.417110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6a8c5650-6128-434d-9be9-84d9833cee7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:17.418301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6a8c5650-6128-434d-9be9-84d9833cee7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:17.421119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 997/1959] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:17 2022] DELETE /identity/v3/projects/d29ce60144344f6e8b24826f0ed0b4bd => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:17.426534 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62f297-2af6-4b0a-aaf7-403cf94673c0 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:17.427070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62f297-2af6-4b0a-aaf7-403cf94673c0 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:17.427277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62f297-2af6-4b0a-aaf7-403cf94673c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:17.471507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6c62f297-2af6-4b0a-aaf7-403cf94673c0 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:17.474535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6c62f297-2af6-4b0a-aaf7-403cf94673c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:17.574459 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 963/1960] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:17.589590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c15dd-301a-40ea-8488-b8af13af34e6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:17.589961 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c15dd-301a-40ea-8488-b8af13af34e6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:17.590393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c15dd-301a-40ea-8488-b8af13af34e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:17.627420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a56c15dd-301a-40ea-8488-b8af13af34e6 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:17.629250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a56c15dd-301a-40ea-8488-b8af13af34e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:17.641163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 998/1961] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:17.645631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62f297-2af6-4b0a-aaf7-403cf94673c0 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:17.660248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62f297-2af6-4b0a-aaf7-403cf94673c0 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:17.667536 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb2e53-55e9-4088-9918-aaea238a0a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:17.668448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb2e53-55e9-4088-9918-aaea238a0a9c admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:17.668793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb2e53-55e9-4088-9918-aaea238a0a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:17.669124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb2e53-55e9-4088-9918-aaea238a0a9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:17.670017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb2e53-55e9-4088-9918-aaea238a0a9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:37:17.670384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb2e53-55e9-4088-9918-aaea238a0a9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:17.672143 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb2e53-55e9-4088-9918-aaea238a0a9c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:17.676131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:17.708245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 964/1962] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:37:17.712440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c15dd-301a-40ea-8488-b8af13af34e6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:17.726432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c15dd-301a-40ea-8488-b8af13af34e6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:17.732469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d201af-bbfc-4c3d-8a8b-a6ac066c59b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:17.733151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d201af-bbfc-4c3d-8a8b-a6ac066c59b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:17.733496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d201af-bbfc-4c3d-8a8b-a6ac066c59b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:17.733749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d201af-bbfc-4c3d-8a8b-a6ac066c59b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:17.734492 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d201af-bbfc-4c3d-8a8b-a6ac066c59b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:17.736672 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d201af-bbfc-4c3d-8a8b-a6ac066c59b6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:17.760803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:17.804400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 999/1963] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:17 2022] POST /identity/v3/projects => generated 369 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:37:17.807166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb2e53-55e9-4088-9918-aaea238a0a9c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:17.840821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb2e53-55e9-4088-9918-aaea238a0a9c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:17.920334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93304364-0b61-47ba-b631-affd6f2b1465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:17.921261 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93304364-0b61-47ba-b631-affd6f2b1465 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:17.921678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93304364-0b61-47ba-b631-affd6f2b1465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:17.922075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93304364-0b61-47ba-b631-affd6f2b1465 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:17.922921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93304364-0b61-47ba-b631-affd6f2b1465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:17.924917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93304364-0b61-47ba-b631-affd6f2b1465 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:17.960239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 965/1964] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:17 2022] POST /identity/v3/users => generated 285 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:17.964627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d201af-bbfc-4c3d-8a8b-a6ac066c59b6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:17.987548 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d201af-bbfc-4c3d-8a8b-a6ac066c59b6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:17.996337 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f35eb-6dff-4fff-b726-456243a9c09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:17.997064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f35eb-6dff-4fff-b726-456243a9c09e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:17.997716 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f35eb-6dff-4fff-b726-456243a9c09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:17.998049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f35eb-6dff-4fff-b726-456243a9c09e admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:17.998773 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f35eb-6dff-4fff-b726-456243a9c09e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:18.000479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f35eb-6dff-4fff-b726-456243a9c09e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:18.025368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1000/1965] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:18.028724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93304364-0b61-47ba-b631-affd6f2b1465 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:18.048031 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93304364-0b61-47ba-b631-affd6f2b1465 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:18.053760 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd08816-b814-4aac-8119-bfe60b18c670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:18.054623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd08816-b814-4aac-8119-bfe60b18c670 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:18.055034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd08816-b814-4aac-8119-bfe60b18c670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:18.055436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd08816-b814-4aac-8119-bfe60b18c670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:18.056395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd08816-b814-4aac-8119-bfe60b18c670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:18.058434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd08816-b814-4aac-8119-bfe60b18c670 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:18.074384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 966/1966] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:18.079853 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f35eb-6dff-4fff-b726-456243a9c09e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:18.095318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f35eb-6dff-4fff-b726-456243a9c09e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:18.108475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6d4c2-a8fa-4207-88d9-87a106207aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:18.109732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6d4c2-a8fa-4207-88d9-87a106207aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:18.110298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6d4c2-a8fa-4207-88d9-87a106207aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:18.110591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6d4c2-a8fa-4207-88d9-87a106207aa3 admin admin] PATH_INFO: `/v3/projects/c85bd65bebf14ec490166c4971f26bf4/users/22861dd3ba6e41c9b9d92122a8f31183/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:18.147059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6d4c2-a8fa-4207-88d9-87a106207aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c85bd65bebf14ec490166c4971f26bf4, user_id=22861dd3ba6e41c9b9d92122a8f31183, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:18.155089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6d4c2-a8fa-4207-88d9-87a106207aa3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:18.208939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1001/1967] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:18 2022] PUT /identity/v3/projects/c85bd65bebf14ec490166c4971f26bf4/users/22861dd3ba6e41c9b9d92122a8f31183/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:18.209473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae9863-7ee4-468a-80d7-ce75103f9b53 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:18.209473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae9863-7ee4-468a-80d7-ce75103f9b53 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:18.209473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae9863-7ee4-468a-80d7-ce75103f9b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:18.272331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-b6ae9863-7ee4-468a-80d7-ce75103f9b53 None None] MFA Rules not processed for user `22861dd3ba6e41c9b9d92122a8f31183`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:18.273573 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b6ae9863-7ee4-468a-80d7-ce75103f9b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:18.301097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6d4c2-a8fa-4207-88d9-87a106207aa3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:18.302975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5dc6d4c2-a8fa-4207-88d9-87a106207aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:18.408940 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6d4c2-a8fa-4207-88d9-87a106207aa3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:18.420015 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b5515-c6db-4b1f-abb3-47bda0bdf17e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:18.420648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b5515-c6db-4b1f-abb3-47bda0bdf17e service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:18.420931 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b5515-c6db-4b1f-abb3-47bda0bdf17e service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:18.423336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b5515-c6db-4b1f-abb3-47bda0bdf17e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:18.449615 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-418b5515-c6db-4b1f-abb3-47bda0bdf17e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:18.489134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 967/1968] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:18.524351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae9863-7ee4-468a-80d7-ce75103f9b53 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:18.527945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b6ae9863-7ee4-468a-80d7-ce75103f9b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:18.640672 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b5515-c6db-4b1f-abb3-47bda0bdf17e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:18.643226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b5515-c6db-4b1f-abb3-47bda0bdf17e service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:18.661184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1002/1969] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:18.711486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:18.733429 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae9863-7ee4-468a-80d7-ce75103f9b53 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:18.740333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916db81-bf42-4fcf-a5c2-d897504279e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfe36a14a11493397e4018b773d3cc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:18.741385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916db81-bf42-4fcf-a5c2-d897504279e6 service glance] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:18.741862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916db81-bf42-4fcf-a5c2-d897504279e6 service glance] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:18.742333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916db81-bf42-4fcf-a5c2-d897504279e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:18.763423 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916db81-bf42-4fcf-a5c2-d897504279e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:18.768024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916db81-bf42-4fcf-a5c2-d897504279e6 service glance] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:18.797844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 968/1970] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:37:19.608513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b5515-c6db-4b1f-abb3-47bda0bdf17e service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:19.609927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-418b5515-c6db-4b1f-abb3-47bda0bdf17e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:19.687041 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916db81-bf42-4fcf-a5c2-d897504279e6 service glance] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:19.697240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916db81-bf42-4fcf-a5c2-d897504279e6 service glance] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:19.701924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759e6f6-6e35-4ae3-89a7-6fd1d6df663f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:19.702673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759e6f6-6e35-4ae3-89a7-6fd1d6df663f service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:19.702990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759e6f6-6e35-4ae3-89a7-6fd1d6df663f service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:19.703649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759e6f6-6e35-4ae3-89a7-6fd1d6df663f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:19.703755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b5515-c6db-4b1f-abb3-47bda0bdf17e service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:19.705656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2759e6f6-6e35-4ae3-89a7-6fd1d6df663f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:19.709318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb154d-35f0-4c2e-b18f-4e4e5f3d04ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:19.709993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb154d-35f0-4c2e-b18f-4e4e5f3d04ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:19.710247 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb154d-35f0-4c2e-b18f-4e4e5f3d04ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:19.710536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb154d-35f0-4c2e-b18f-4e4e5f3d04ff admin admin] PATH_INFO: `/v3/users/77cd5fc8621e4d879016ffc6639ab5c6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:19.711837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb154d-35f0-4c2e-b18f-4e4e5f3d04ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=77cd5fc8621e4d879016ffc6639ab5c6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:19.715111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb154d-35f0-4c2e-b18f-4e4e5f3d04ff admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:19.731772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:19.775669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-58eb154d-35f0-4c2e-b18f-4e4e5f3d04ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77cd5fc8621e4d879016ffc6639ab5c6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:19.784855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-58eb154d-35f0-4c2e-b18f-4e4e5f3d04ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77cd5fc8621e4d879016ffc6639ab5c6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:19.785602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-58eb154d-35f0-4c2e-b18f-4e4e5f3d04ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77cd5fc8621e4d879016ffc6639ab5c6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:19.786046 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759e6f6-6e35-4ae3-89a7-6fd1d6df663f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:19.787674 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759e6f6-6e35-4ae3-89a7-6fd1d6df663f service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:19.798151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 969/1971] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:19.806264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-58eb154d-35f0-4c2e-b18f-4e4e5f3d04ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77cd5fc8621e4d879016ffc6639ab5c6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:19.814688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-58eb154d-35f0-4c2e-b18f-4e4e5f3d04ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77cd5fc8621e4d879016ffc6639ab5c6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:19.821223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1003/1972] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:19 2022] DELETE /identity/v3/users/77cd5fc8621e4d879016ffc6639ab5c6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:19.825776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46137f8c-67ce-4413-873f-a283a12741d9 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:19.826064 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46137f8c-67ce-4413-873f-a283a12741d9 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:19.826310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46137f8c-67ce-4413-873f-a283a12741d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:19.847470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-46137f8c-67ce-4413-873f-a283a12741d9 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:19.848717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-46137f8c-67ce-4413-873f-a283a12741d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:19.927002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 970/1973] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:19.937732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb154d-35f0-4c2e-b18f-4e4e5f3d04ff admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:19.939477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-58eb154d-35f0-4c2e-b18f-4e4e5f3d04ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:20.016347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb154d-35f0-4c2e-b18f-4e4e5f3d04ff admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:20.020309 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a3c99-f37d-4726-84c7-c28b390c2a75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:20.021001 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a3c99-f37d-4726-84c7-c28b390c2a75 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:20.021285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a3c99-f37d-4726-84c7-c28b390c2a75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:20.021555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a3c99-f37d-4726-84c7-c28b390c2a75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:20.031949 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a3c99-f37d-4726-84c7-c28b390c2a75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:20.033486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a3c99-f37d-4726-84c7-c28b390c2a75 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:20.044578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1004/1974] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:20.386690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46137f8c-67ce-4413-873f-a283a12741d9 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:20.388291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-46137f8c-67ce-4413-873f-a283a12741d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:20.401235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46137f8c-67ce-4413-873f-a283a12741d9 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:20.406506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae90ea-c6a8-4471-88f0-c38430108590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:20.407345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae90ea-c6a8-4471-88f0-c38430108590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:20.407880 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae90ea-c6a8-4471-88f0-c38430108590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:20.408328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae90ea-c6a8-4471-88f0-c38430108590 admin admin] PATH_INFO: `/v3/projects/b6185130f1c64c8da2923854ef7bad1c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:20.410227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae90ea-c6a8-4471-88f0-c38430108590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6185130f1c64c8da2923854ef7bad1c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:20.412474 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae90ea-c6a8-4471-88f0-c38430108590 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:20.471512 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-43ae90ea-c6a8-4471-88f0-c38430108590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6185130f1c64c8da2923854ef7bad1c', 'request_id': 'req-43ae90ea-c6a8-4471-88f0-c38430108590'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:20.481239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-43ae90ea-c6a8-4471-88f0-c38430108590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6185130f1c64c8da2923854ef7bad1c', 'request_id': 'req-43ae90ea-c6a8-4471-88f0-c38430108590'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:20.487578 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-43ae90ea-c6a8-4471-88f0-c38430108590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6185130f1c64c8da2923854ef7bad1c', 'request_id': 'req-43ae90ea-c6a8-4471-88f0-c38430108590'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:20.501588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-43ae90ea-c6a8-4471-88f0-c38430108590 admin admin] The token cache is being invalidate because project b6185130f1c64c8da2923854ef7bad1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:20.501887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-43ae90ea-c6a8-4471-88f0-c38430108590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:20.502634 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-43ae90ea-c6a8-4471-88f0-c38430108590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:20.507262 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 971/1975] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:20 2022] DELETE /identity/v3/projects/b6185130f1c64c8da2923854ef7bad1c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:20.512721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d94ef-906c-4c9e-a848-a8d6ccbcdcae None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:20.512721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d94ef-906c-4c9e-a848-a8d6ccbcdcae None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:20.512721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d94ef-906c-4c9e-a848-a8d6ccbcdcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:20.535681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f36d94ef-906c-4c9e-a848-a8d6ccbcdcae None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:20.537003 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f36d94ef-906c-4c9e-a848-a8d6ccbcdcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:20.621120 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1005/1976] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:20.629801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724beb97-236c-4c20-b337-0bc17bff64e9 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:20.630148 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724beb97-236c-4c20-b337-0bc17bff64e9 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:20.630340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724beb97-236c-4c20-b337-0bc17bff64e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:20.643737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:20.684922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-724beb97-236c-4c20-b337-0bc17bff64e9 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:20.686560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-724beb97-236c-4c20-b337-0bc17bff64e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:20.695827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 972/1977] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:20.700887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d94ef-906c-4c9e-a848-a8d6ccbcdcae None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:20.714061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d94ef-906c-4c9e-a848-a8d6ccbcdcae None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:20.720321 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9ed90-dc60-407f-a734-0bbf4b7e969a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:20.720996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9ed90-dc60-407f-a734-0bbf4b7e969a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:20.721351 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9ed90-dc60-407f-a734-0bbf4b7e969a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:20.721703 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9ed90-dc60-407f-a734-0bbf4b7e969a admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:20.722484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9ed90-dc60-407f-a734-0bbf4b7e969a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:37:20.722875 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9ed90-dc60-407f-a734-0bbf4b7e969a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:20.724777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9ed90-dc60-407f-a734-0bbf4b7e969a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:20.751937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1006/1978] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:37:20.754869 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724beb97-236c-4c20-b337-0bc17bff64e9 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:20.765656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724beb97-236c-4c20-b337-0bc17bff64e9 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:20.778093 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a283f-5458-49a5-8a55-1457acd20a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:20.778666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a283f-5458-49a5-8a55-1457acd20a9a admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:20.778942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a283f-5458-49a5-8a55-1457acd20a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:20.779212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a283f-5458-49a5-8a55-1457acd20a9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:20.779869 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a283f-5458-49a5-8a55-1457acd20a9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:20.781845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a283f-5458-49a5-8a55-1457acd20a9a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:20.813856 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 973/1979] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:20 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:20.817833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9ed90-dc60-407f-a734-0bbf4b7e969a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:20.831012 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9ed90-dc60-407f-a734-0bbf4b7e969a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:20.854273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:20.897163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b591d-df94-46da-9681-4c7b77b6fb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:20.897818 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b591d-df94-46da-9681-4c7b77b6fb99 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:20.898091 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b591d-df94-46da-9681-4c7b77b6fb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:20.898368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b591d-df94-46da-9681-4c7b77b6fb99 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:20.899000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b591d-df94-46da-9681-4c7b77b6fb99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:20.901926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b591d-df94-46da-9681-4c7b77b6fb99 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:20.928609 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1007/1980] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:20 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:20.933540 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a283f-5458-49a5-8a55-1457acd20a9a admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:20.945902 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a283f-5458-49a5-8a55-1457acd20a9a admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:20.952285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca259ce-abfc-4254-a6f3-e46a13bb9a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:20.954086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca259ce-abfc-4254-a6f3-e46a13bb9a3f admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:20.954701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca259ce-abfc-4254-a6f3-e46a13bb9a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:20.954701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca259ce-abfc-4254-a6f3-e46a13bb9a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:20.955229 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca259ce-abfc-4254-a6f3-e46a13bb9a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:20.957244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca259ce-abfc-4254-a6f3-e46a13bb9a3f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:20.978115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 974/1981] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:20.982985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b591d-df94-46da-9681-4c7b77b6fb99 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:20.999198 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b591d-df94-46da-9681-4c7b77b6fb99 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:21.008595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd8d4e-2771-4095-8f34-83f4817ced4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:21.009229 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd8d4e-2771-4095-8f34-83f4817ced4b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:21.009508 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd8d4e-2771-4095-8f34-83f4817ced4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:21.009767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd8d4e-2771-4095-8f34-83f4817ced4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:21.010420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd8d4e-2771-4095-8f34-83f4817ced4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:21.012087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd8d4e-2771-4095-8f34-83f4817ced4b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:21.026754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1008/1982] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:37:21.027931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca259ce-abfc-4254-a6f3-e46a13bb9a3f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:21.038035 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca259ce-abfc-4254-a6f3-e46a13bb9a3f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:21.044405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4a5c9-b62c-4f2d-a5e0-e9f968188405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:21.045103 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4a5c9-b62c-4f2d-a5e0-e9f968188405 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:21.045383 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4a5c9-b62c-4f2d-a5e0-e9f968188405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:21.045651 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4a5c9-b62c-4f2d-a5e0-e9f968188405 admin admin] PATH_INFO: `/v3/projects/d5df463e3a1640ec8041d2b4bb6311f2/users/da359f520a9d411eb52da5700c07f8d7/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:21.062699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4a5c9-b62c-4f2d-a5e0-e9f968188405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5df463e3a1640ec8041d2b4bb6311f2, user_id=da359f520a9d411eb52da5700c07f8d7, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:21.065860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4a5c9-b62c-4f2d-a5e0-e9f968188405 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:21.084775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 975/1983] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:21 2022] PUT /identity/v3/projects/d5df463e3a1640ec8041d2b4bb6311f2/users/da359f520a9d411eb52da5700c07f8d7/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:21.094107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1cfeb-7201-496c-adbb-9970ad55dd72 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:21.094394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1cfeb-7201-496c-adbb-9970ad55dd72 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:21.094665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1cfeb-7201-496c-adbb-9970ad55dd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:21.119536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8df1cfeb-7201-496c-adbb-9970ad55dd72 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:21.121135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8df1cfeb-7201-496c-adbb-9970ad55dd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:21.194459 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1009/1984] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:21.207460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4a5c9-b62c-4f2d-a5e0-e9f968188405 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:21.210111 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-fba4a5c9-b62c-4f2d-a5e0-e9f968188405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:21.282565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4a5c9-b62c-4f2d-a5e0-e9f968188405 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:21.286321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606e841-b69c-47b1-9aec-c149d50ede45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:21.286938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e841-b69c-47b1-9aec-c149d50ede45 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:21.287215 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e841-b69c-47b1-9aec-c149d50ede45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:21.287486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e841-b69c-47b1-9aec-c149d50ede45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:21.297571 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606e841-b69c-47b1-9aec-c149d50ede45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:21.299046 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606e841-b69c-47b1-9aec-c149d50ede45 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:21.309490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 976/1985] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:23.957234 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1cfeb-7201-496c-adbb-9970ad55dd72 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:23.961905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8df1cfeb-7201-496c-adbb-9970ad55dd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:23.979089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1cfeb-7201-496c-adbb-9970ad55dd72 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:23.994383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e7685-7feb-4b39-9251-2c2c8d92b56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:23.996204 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e7685-7feb-4b39-9251-2c2c8d92b56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:23.996548 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e7685-7feb-4b39-9251-2c2c8d92b56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:23.996825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e7685-7feb-4b39-9251-2c2c8d92b56d admin admin] PATH_INFO: `/v3/users/564e343f84cf4ab89381bb2a79e92a8e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:23.998099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e7685-7feb-4b39-9251-2c2c8d92b56d admin admin] RBAC: Authorizing `identity:delete_user(user_id=564e343f84cf4ab89381bb2a79e92a8e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:24.001225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e7685-7feb-4b39-9251-2c2c8d92b56d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:24.088814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-863e7685-7feb-4b39-9251-2c2c8d92b56d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '564e343f84cf4ab89381bb2a79e92a8e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:24.100701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-863e7685-7feb-4b39-9251-2c2c8d92b56d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '564e343f84cf4ab89381bb2a79e92a8e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:24.103420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-863e7685-7feb-4b39-9251-2c2c8d92b56d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '564e343f84cf4ab89381bb2a79e92a8e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:24.133305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-863e7685-7feb-4b39-9251-2c2c8d92b56d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '564e343f84cf4ab89381bb2a79e92a8e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:24.151746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-863e7685-7feb-4b39-9251-2c2c8d92b56d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '564e343f84cf4ab89381bb2a79e92a8e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:24.171996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1010/1986] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:23 2022] DELETE /identity/v3/users/564e343f84cf4ab89381bb2a79e92a8e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:24.179687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92890464-73cb-4666-8d10-9b0d7f1cf417 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:24.180798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92890464-73cb-4666-8d10-9b0d7f1cf417 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:24.181274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92890464-73cb-4666-8d10-9b0d7f1cf417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:24.221478 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-92890464-73cb-4666-8d10-9b0d7f1cf417 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:24.223024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-92890464-73cb-4666-8d10-9b0d7f1cf417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:24.308029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 977/1987] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:24.317248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e7685-7feb-4b39-9251-2c2c8d92b56d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:24.319855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-863e7685-7feb-4b39-9251-2c2c8d92b56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:24.353995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:24.418049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e7685-7feb-4b39-9251-2c2c8d92b56d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:24.423880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377801b5-270b-43c5-b81d-30abe99805fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:24.424724 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377801b5-270b-43c5-b81d-30abe99805fc service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:24.425095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377801b5-270b-43c5-b81d-30abe99805fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:24.425456 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377801b5-270b-43c5-b81d-30abe99805fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:24.440759 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377801b5-270b-43c5-b81d-30abe99805fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:24.443431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377801b5-270b-43c5-b81d-30abe99805fc service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:24.466214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1011/1988] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:25.118670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92890464-73cb-4666-8d10-9b0d7f1cf417 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:25.121019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-92890464-73cb-4666-8d10-9b0d7f1cf417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:25.140268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92890464-73cb-4666-8d10-9b0d7f1cf417 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:25.147155 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:25.148253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:25.148636 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:25.149068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] PATH_INFO: `/v3/projects/b27d950768744621a54402eb5fdc247c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:25.150351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b27d950768744621a54402eb5fdc247c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:25.160208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:25.220893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:25.231416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b27d950768744621a54402eb5fdc247c', 'request_id': 'req-9bc28ba1-778f-43e6-9dec-92c63f697bf7'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:25.246135 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b27d950768744621a54402eb5fdc247c', 'request_id': 'req-9bc28ba1-778f-43e6-9dec-92c63f697bf7'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:25.254000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b27d950768744621a54402eb5fdc247c', 'request_id': 'req-9bc28ba1-778f-43e6-9dec-92c63f697bf7'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:25.269985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] The token cache is being invalidate because project b27d950768744621a54402eb5fdc247c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:25.270100 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:25.270926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:25.273077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 978/1989] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:25 2022] DELETE /identity/v3/projects/b27d950768744621a54402eb5fdc247c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:25.280291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377801b5-270b-43c5-b81d-30abe99805fc service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:25.293123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377801b5-270b-43c5-b81d-30abe99805fc service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:25.405192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6c142-056c-46b6-bc3e-e000cfe2c392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:25.406007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6c142-056c-46b6-bc3e-e000cfe2c392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:25.406383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6c142-056c-46b6-bc3e-e000cfe2c392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:25.406757 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6c142-056c-46b6-bc3e-e000cfe2c392 admin admin] PATH_INFO: `/v3/users/3c66c46d8d8c4e268e1c01ff24ead685` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:25.408775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6c142-056c-46b6-bc3e-e000cfe2c392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c66c46d8d8c4e268e1c01ff24ead685)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:25.411118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6c142-056c-46b6-bc3e-e000cfe2c392 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:25.522404 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cff6c142-056c-46b6-bc3e-e000cfe2c392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c66c46d8d8c4e268e1c01ff24ead685'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:25.536436 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cff6c142-056c-46b6-bc3e-e000cfe2c392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c66c46d8d8c4e268e1c01ff24ead685'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:25.538134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cff6c142-056c-46b6-bc3e-e000cfe2c392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c66c46d8d8c4e268e1c01ff24ead685'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:25.556056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cff6c142-056c-46b6-bc3e-e000cfe2c392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c66c46d8d8c4e268e1c01ff24ead685'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:25.565509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cff6c142-056c-46b6-bc3e-e000cfe2c392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c66c46d8d8c4e268e1c01ff24ead685'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:25.583084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1012/1990] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:25 2022] DELETE /identity/v3/users/3c66c46d8d8c4e268e1c01ff24ead685 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:26.105841 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:26.109741 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:26.147618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc28ba1-778f-43e6-9dec-92c63f697bf7 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:26.300942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:26.302431 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:26.303759 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:26.304212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] PATH_INFO: `/v3/projects/24ff013d096b4b96aee724255343b972` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:26.305977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24ff013d096b4b96aee724255343b972)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:26.308641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:26.419490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24ff013d096b4b96aee724255343b972', 'request_id': 'req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:26.433054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24ff013d096b4b96aee724255343b972', 'request_id': 'req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:26.452090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24ff013d096b4b96aee724255343b972', 'request_id': 'req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:26.476796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] The token cache is being invalidate because project 24ff013d096b4b96aee724255343b972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:26.477623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:26.478562 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:26.479475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc19266-0536-41d8-8b70-a8238007f11a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:26.480035 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc19266-0536-41d8-8b70-a8238007f11a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:26.480035 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc19266-0536-41d8-8b70-a8238007f11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:26.481016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 979/1991] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:26 2022] DELETE /identity/v3/projects/24ff013d096b4b96aee724255343b972 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:26.485767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:26.494431 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:26.551072 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cfc19266-0536-41d8-8b70-a8238007f11a None None] MFA Rules not processed for user `da359f520a9d411eb52da5700c07f8d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:26.552420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cfc19266-0536-41d8-8b70-a8238007f11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:26.584826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:26.592052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:26.593575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5edcc8-e93e-4a23-bf02-f8b8b4a6f681 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:26.605631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c729b-d981-4fa3-b5ca-f1e4291c11b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:26.606831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c729b-d981-4fa3-b5ca-f1e4291c11b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:26.607267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c729b-d981-4fa3-b5ca-f1e4291c11b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:26.607677 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c729b-d981-4fa3-b5ca-f1e4291c11b7 admin admin] PATH_INFO: `/v3/users/3741d039853d418d9bd12760255d66e4` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:26.609532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c729b-d981-4fa3-b5ca-f1e4291c11b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3741d039853d418d9bd12760255d66e4)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:26.614765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c729b-d981-4fa3-b5ca-f1e4291c11b7 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:26.649120 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1013/1992] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:26.655748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc19266-0536-41d8-8b70-a8238007f11a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:26.656273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cfc19266-0536-41d8-8b70-a8238007f11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:26.686805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc19266-0536-41d8-8b70-a8238007f11a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:26.690905 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-626c729b-d981-4fa3-b5ca-f1e4291c11b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3741d039853d418d9bd12760255d66e4'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:26.691826 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9676102-4f2c-45e2-be31-cd46d8545eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:26.692621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9676102-4f2c-45e2-be31-cd46d8545eac admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:26.692998 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9676102-4f2c-45e2-be31-cd46d8545eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:26.693357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9676102-4f2c-45e2-be31-cd46d8545eac admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:26.694128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9676102-4f2c-45e2-be31-cd46d8545eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:26.696319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9676102-4f2c-45e2-be31-cd46d8545eac admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:26.707431 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-626c729b-d981-4fa3-b5ca-f1e4291c11b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3741d039853d418d9bd12760255d66e4'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:26.719159 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-626c729b-d981-4fa3-b5ca-f1e4291c11b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3741d039853d418d9bd12760255d66e4'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:26.734724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-626c729b-d981-4fa3-b5ca-f1e4291c11b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3741d039853d418d9bd12760255d66e4'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:26.735732 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-626c729b-d981-4fa3-b5ca-f1e4291c11b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3741d039853d418d9bd12760255d66e4'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:26.742606 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 980/1993] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:26 2022] DELETE /identity/v3/users/3741d039853d418d9bd12760255d66e4 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:26.743731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1014/1994] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:26 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:37:26.749777 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c729b-d981-4fa3-b5ca-f1e4291c11b7 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:26.758618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-626c729b-d981-4fa3-b5ca-f1e4291c11b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:26.878877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c729b-d981-4fa3-b5ca-f1e4291c11b7 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:26.887232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2a0b9-83b9-4c1b-b652-d5c43d35b462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:26.888427 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2a0b9-83b9-4c1b-b652-d5c43d35b462 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:26.888688 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2a0b9-83b9-4c1b-b652-d5c43d35b462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:26.889114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2a0b9-83b9-4c1b-b652-d5c43d35b462 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:26.890164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2a0b9-83b9-4c1b-b652-d5c43d35b462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:26.893543 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2a0b9-83b9-4c1b-b652-d5c43d35b462 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:26.946712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 981/1995] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:26 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:26.947874 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9676102-4f2c-45e2-be31-cd46d8545eac admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:27.078705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9676102-4f2c-45e2-be31-cd46d8545eac admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:27.084311 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f12d17-b0c6-4423-8574-30fd45fcf2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:27.085463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f12d17-b0c6-4423-8574-30fd45fcf2ae admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:27.086081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f12d17-b0c6-4423-8574-30fd45fcf2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:27.086515 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f12d17-b0c6-4423-8574-30fd45fcf2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:27.087292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f12d17-b0c6-4423-8574-30fd45fcf2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:27.089185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f12d17-b0c6-4423-8574-30fd45fcf2ae admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:27.106888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1015/1996] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:27.113704 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2a0b9-83b9-4c1b-b652-d5c43d35b462 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:27.128783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2a0b9-83b9-4c1b-b652-d5c43d35b462 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:27.141332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada69852-7247-444a-9989-16e45040cce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:27.143582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada69852-7247-444a-9989-16e45040cce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:27.144370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada69852-7247-444a-9989-16e45040cce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:27.146288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada69852-7247-444a-9989-16e45040cce9 admin admin] PATH_INFO: `/v3/projects/ad365a4e010f4e1ca916cdd9b72e5065/users/c4c610da9afb47cab01325b1dd637848/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:27.154473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f12d17-b0c6-4423-8574-30fd45fcf2ae admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:27.157086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-34f12d17-b0c6-4423-8574-30fd45fcf2ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:27.183314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f12d17-b0c6-4423-8574-30fd45fcf2ae admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:27.184839 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada69852-7247-444a-9989-16e45040cce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad365a4e010f4e1ca916cdd9b72e5065, user_id=c4c610da9afb47cab01325b1dd637848, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:27.188367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada69852-7247-444a-9989-16e45040cce9 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:27.202681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:27.203662 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:27.204055 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:27.204342 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75 admin admin] PATH_INFO: `/v3/projects/271d36fe86964d77b0e8c6d7f1caff61` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:27.205893 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=271d36fe86964d77b0e8c6d7f1caff61)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:27.208487 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:27.225226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 982/1997] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:27 2022] PUT /identity/v3/projects/ad365a4e010f4e1ca916cdd9b72e5065/users/c4c610da9afb47cab01325b1dd637848/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:27.226320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada69852-7247-444a-9989-16e45040cce9 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:27.278558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '271d36fe86964d77b0e8c6d7f1caff61', 'request_id': 'req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:27.290594 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '271d36fe86964d77b0e8c6d7f1caff61', 'request_id': 'req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:27.298826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:27.299075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '271d36fe86964d77b0e8c6d7f1caff61', 'request_id': 'req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:27.319396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75 admin admin] The token cache is being invalidate because project 271d36fe86964d77b0e8c6d7f1caff61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:27.319686 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:27.320413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4ef7f4d6-1cd0-45c6-a25e-952ad15f1e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:27.322132 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1016/1998] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:27 2022] DELETE /identity/v3/projects/271d36fe86964d77b0e8c6d7f1caff61 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:27.328691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49afa41-725e-46e4-9adc-ef7c1dc94f2a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:27.329130 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49afa41-725e-46e4-9adc-ef7c1dc94f2a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:27.329652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49afa41-725e-46e4-9adc-ef7c1dc94f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:27.332624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada69852-7247-444a-9989-16e45040cce9 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:27.358444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e49afa41-725e-46e4-9adc-ef7c1dc94f2a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:27.360448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e49afa41-725e-46e4-9adc-ef7c1dc94f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:27.395949 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:27.411804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf3de1-7c80-413e-8427-07f48296f4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:27.413054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf3de1-7c80-413e-8427-07f48296f4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:27.413586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf3de1-7c80-413e-8427-07f48296f4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:27.413954 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf3de1-7c80-413e-8427-07f48296f4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:27.415125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf3de1-7c80-413e-8427-07f48296f4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:27.417469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf3de1-7c80-413e-8427-07f48296f4d4 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:27.431372 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 983/1999] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:27.436272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf3de1-7c80-413e-8427-07f48296f4d4 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:27.441816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-aacf3de1-7c80-413e-8427-07f48296f4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:27.457415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf3de1-7c80-413e-8427-07f48296f4d4 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:27.465024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311321c-b1d5-48af-a92f-9178633226fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:27.465911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311321c-b1d5-48af-a92f-9178633226fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:27.466610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311321c-b1d5-48af-a92f-9178633226fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:27.466978 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311321c-b1d5-48af-a92f-9178633226fc admin admin] PATH_INFO: `/v3/projects/ad365a4e010f4e1ca916cdd9b72e5065/users/c4c610da9afb47cab01325b1dd637848/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:27.471145 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311321c-b1d5-48af-a92f-9178633226fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad365a4e010f4e1ca916cdd9b72e5065, user_id=c4c610da9afb47cab01325b1dd637848, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:27.476786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311321c-b1d5-48af-a92f-9178633226fc admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:27.491984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 984/2000] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:27 2022] PUT /identity/v3/projects/ad365a4e010f4e1ca916cdd9b72e5065/users/c4c610da9afb47cab01325b1dd637848/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:27.507282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1017/2001] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:37:27.517457 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd93f3-d616-4201-beae-8d80004d4250 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:27.517713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd93f3-d616-4201-beae-8d80004d4250 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:27.518059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd93f3-d616-4201-beae-8d80004d4250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:27.542807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-f3bd93f3-d616-4201-beae-8d80004d4250 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:27.544107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f3bd93f3-d616-4201-beae-8d80004d4250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:27.642319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 985/2002] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:27.645223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49afa41-725e-46e4-9adc-ef7c1dc94f2a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:27.664988 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49afa41-725e-46e4-9adc-ef7c1dc94f2a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:27.671445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d6550-254f-4d40-8f4c-796803f52a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:27.676564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d6550-254f-4d40-8f4c-796803f52a54 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:27.676870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d6550-254f-4d40-8f4c-796803f52a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:27.677323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d6550-254f-4d40-8f4c-796803f52a54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:27.678376 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d6550-254f-4d40-8f4c-796803f52a54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:37:27.678695 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d6550-254f-4d40-8f4c-796803f52a54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:27.680427 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d6550-254f-4d40-8f4c-796803f52a54 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:27.704189 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1018/2003] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:37:27.705445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd93f3-d616-4201-beae-8d80004d4250 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:27.721300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd93f3-d616-4201-beae-8d80004d4250 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:27.727783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2edbd4-b3d0-46a4-a21f-4a9a4b36e877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:27.728570 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edbd4-b3d0-46a4-a21f-4a9a4b36e877 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:27.730588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edbd4-b3d0-46a4-a21f-4a9a4b36e877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:27.730588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edbd4-b3d0-46a4-a21f-4a9a4b36e877 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:27.730588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2edbd4-b3d0-46a4-a21f-4a9a4b36e877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:27.732448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2edbd4-b3d0-46a4-a21f-4a9a4b36e877 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:27.821194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 986/2004] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:27 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:27.826025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d6550-254f-4d40-8f4c-796803f52a54 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:27.859333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d6550-254f-4d40-8f4c-796803f52a54 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:28.019144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20574bc-1762-4ebf-8ac3-9438ea885921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:28.020568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20574bc-1762-4ebf-8ac3-9438ea885921 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:28.021292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20574bc-1762-4ebf-8ac3-9438ea885921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:28.022202 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20574bc-1762-4ebf-8ac3-9438ea885921 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:28.023653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20574bc-1762-4ebf-8ac3-9438ea885921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:28.028486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20574bc-1762-4ebf-8ac3-9438ea885921 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:28.080965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1019/2005] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:27 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:28.082013 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2edbd4-b3d0-46a4-a21f-4a9a4b36e877 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:28.097680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2edbd4-b3d0-46a4-a21f-4a9a4b36e877 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:28.111482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053a05d-65fe-46e1-9d93-5fd628980bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:28.112176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053a05d-65fe-46e1-9d93-5fd628980bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:28.112454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053a05d-65fe-46e1-9d93-5fd628980bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:28.113202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053a05d-65fe-46e1-9d93-5fd628980bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:28.113997 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053a05d-65fe-46e1-9d93-5fd628980bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:28.116733 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053a05d-65fe-46e1-9d93-5fd628980bd0 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:28.133853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 987/2006] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:28.135323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20574bc-1762-4ebf-8ac3-9438ea885921 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:28.147727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20574bc-1762-4ebf-8ac3-9438ea885921 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:28.153076 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c58bba-a9c3-47d8-9d4a-cb4ce82f8c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:28.153862 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c58bba-a9c3-47d8-9d4a-cb4ce82f8c8f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:28.154139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c58bba-a9c3-47d8-9d4a-cb4ce82f8c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:28.154402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c58bba-a9c3-47d8-9d4a-cb4ce82f8c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:28.155459 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c58bba-a9c3-47d8-9d4a-cb4ce82f8c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:28.157592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c58bba-a9c3-47d8-9d4a-cb4ce82f8c8f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:28.173298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1020/2007] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:37:28.177887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053a05d-65fe-46e1-9d93-5fd628980bd0 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:28.197586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053a05d-65fe-46e1-9d93-5fd628980bd0 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:28.206280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc61a3-e366-4308-8e23-f4895a8da400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:28.206280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc61a3-e366-4308-8e23-f4895a8da400 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:28.206280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc61a3-e366-4308-8e23-f4895a8da400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:28.206280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc61a3-e366-4308-8e23-f4895a8da400 admin admin] PATH_INFO: `/v3/projects/45f669183769418a91b495a85c37d4c3/users/64357433676a45a9b6cc2ee2b19f33a1/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:28.245471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc61a3-e366-4308-8e23-f4895a8da400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f669183769418a91b495a85c37d4c3, user_id=64357433676a45a9b6cc2ee2b19f33a1, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:28.252256 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc61a3-e366-4308-8e23-f4895a8da400 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:28.285303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 988/2008] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:28 2022] PUT /identity/v3/projects/45f669183769418a91b495a85c37d4c3/users/64357433676a45a9b6cc2ee2b19f33a1/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:28.296680 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff92bd-b1ef-4079-92f8-b44e4561c691 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:28.296958 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff92bd-b1ef-4079-92f8-b44e4561c691 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:28.297240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff92bd-b1ef-4079-92f8-b44e4561c691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:28.348064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-19ff92bd-b1ef-4079-92f8-b44e4561c691 None None] MFA Rules not processed for user `64357433676a45a9b6cc2ee2b19f33a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:28.348064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-19ff92bd-b1ef-4079-92f8-b44e4561c691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:28.468775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1021/2009] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:28.482232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc61a3-e366-4308-8e23-f4895a8da400 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:28.484072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f8dc61a3-e366-4308-8e23-f4895a8da400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:28.515552 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:28.579738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc61a3-e366-4308-8e23-f4895a8da400 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:28.588720 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55637d9-094b-4ec8-9139-7f207a0d1877 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfe36a14a11493397e4018b773d3cc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:28.588951 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55637d9-094b-4ec8-9139-7f207a0d1877 service glance] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:28.589246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55637d9-094b-4ec8-9139-7f207a0d1877 service glance] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:28.589465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55637d9-094b-4ec8-9139-7f207a0d1877 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:28.609815 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55637d9-094b-4ec8-9139-7f207a0d1877 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:28.609815 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55637d9-094b-4ec8-9139-7f207a0d1877 service glance] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:28.635304 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 989/2010] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:37:34.280299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61089251-4fc7-4631-a3f7-9d23e322a7ea None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:34.281632 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61089251-4fc7-4631-a3f7-9d23e322a7ea None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:34.282062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61089251-4fc7-4631-a3f7-9d23e322a7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:34.363458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-61089251-4fc7-4631-a3f7-9d23e322a7ea None None] MFA Rules not processed for user `c4c610da9afb47cab01325b1dd637848`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:34.365796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-61089251-4fc7-4631-a3f7-9d23e322a7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:34.388464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:34.495910 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1022/2011] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:34.519709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55637d9-094b-4ec8-9139-7f207a0d1877 service glance] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:34.523839 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e55637d9-094b-4ec8-9139-7f207a0d1877 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:34.775110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55637d9-094b-4ec8-9139-7f207a0d1877 service glance] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:34.784894 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88996f-2f14-423a-848a-1ecb09802fb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:34.786184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88996f-2f14-423a-848a-1ecb09802fb4 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:34.786787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88996f-2f14-423a-848a-1ecb09802fb4 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:34.787388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88996f-2f14-423a-848a-1ecb09802fb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:34.793745 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2d88996f-2f14-423a-848a-1ecb09802fb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:34.922303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88996f-2f14-423a-848a-1ecb09802fb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:34.934715 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88996f-2f14-423a-848a-1ecb09802fb4 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:34.975309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 990/2012] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:36.405098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cdffc-499e-44bf-b722-1e502f3613b3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:36.405677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cdffc-499e-44bf-b722-1e502f3613b3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:36.405677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cdffc-499e-44bf-b722-1e502f3613b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:36.478973 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-889cdffc-499e-44bf-b722-1e502f3613b3 None None] MFA Rules not processed for user `da359f520a9d411eb52da5700c07f8d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:36.487749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-889cdffc-499e-44bf-b722-1e502f3613b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:36.522064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1023/2013] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:38.187644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6ae08-d703-4ff5-b8f1-fcae50b57527 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:38.189215 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6ae08-d703-4ff5-b8f1-fcae50b57527 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:38.189663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6ae08-d703-4ff5-b8f1-fcae50b57527 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:38.191576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 991/2014] 10.222.0.62 () {54 vars in 921 bytes} [Thu Sep 1 16:37:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:37:38.530579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceefcc1-31a1-43aa-9de5-e27673fe2e67 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:38.531729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceefcc1-31a1-43aa-9de5-e27673fe2e67 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:38.532433 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceefcc1-31a1-43aa-9de5-e27673fe2e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:38.572633 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7ceefcc1-31a1-43aa-9de5-e27673fe2e67 None None] MFA Rules not processed for user `6281dcf2927a4ed2870125df8801be78`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:38.574225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7ceefcc1-31a1-43aa-9de5-e27673fe2e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:38.614647 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6ae08-d703-4ff5-b8f1-fcae50b57527 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:38.616822 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5dc6ae08-d703-4ff5-b8f1-fcae50b57527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:38.690726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1024/2015] 10.222.0.62 () {58 vars in 1018 bytes} [Thu Sep 1 16:37:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:38.704910 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6d7b3-1508-452c-93ac-4dd385a384b1 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:38.705391 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6d7b3-1508-452c-93ac-4dd385a384b1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:38.705784 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6d7b3-1508-452c-93ac-4dd385a384b1 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:38.707447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1025/2016] 10.222.0.62 () {54 vars in 961 bytes} [Thu Sep 1 16:37:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:37:38.713294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fb842-9a85-420f-be06-c667c31d8aab None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:38.713740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fb842-9a85-420f-be06-c667c31d8aab None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:38.714013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fb842-9a85-420f-be06-c667c31d8aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:38.738760 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-145fb842-9a85-420f-be06-c667c31d8aab None None] MFA Rules not processed for user `6281dcf2927a4ed2870125df8801be78`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:38.740092 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-145fb842-9a85-420f-be06-c667c31d8aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:38.744052 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:38.752665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1026/2017] 10.222.0.62 () {58 vars in 1058 bytes} [Thu Sep 1 16:37:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:38.755245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6ae08-d703-4ff5-b8f1-fcae50b57527 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:38.761073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51d2e7-e8f3-4e5c-8238-d6edf153bced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:38.761807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51d2e7-e8f3-4e5c-8238-d6edf153bced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:38.762111 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51d2e7-e8f3-4e5c-8238-d6edf153bced admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:38.762387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51d2e7-e8f3-4e5c-8238-d6edf153bced admin admin] PATH_INFO: `/v3/users/f21b1fea950d48ef9a55cded8ed172a6` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:38.763569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51d2e7-e8f3-4e5c-8238-d6edf153bced admin admin] RBAC: Authorizing `identity:delete_user(user_id=f21b1fea950d48ef9a55cded8ed172a6)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:38.765406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fb842-9a85-420f-be06-c667c31d8aab None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:38.765749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51d2e7-e8f3-4e5c-8238-d6edf153bced admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:38.776945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fb842-9a85-420f-be06-c667c31d8aab None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:38.783147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746da9c-e03f-4f33-86a8-90eca539b570 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6281dcf2927a4ed2870125df8801be78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:38.783827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746da9c-e03f-4f33-86a8-90eca539b570 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:38.784149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746da9c-e03f-4f33-86a8-90eca539b570 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:38.784427 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746da9c-e03f-4f33-86a8-90eca539b570 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:38.794422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746da9c-e03f-4f33-86a8-90eca539b570 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:38.796240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746da9c-e03f-4f33-86a8-90eca539b570 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:38.808930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1027/2018] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:37:38.826678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6a51d2e7-e8f3-4e5c-8238-d6edf153bced admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f21b1fea950d48ef9a55cded8ed172a6'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:38.842227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6a51d2e7-e8f3-4e5c-8238-d6edf153bced admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f21b1fea950d48ef9a55cded8ed172a6'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:38.849815 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6a51d2e7-e8f3-4e5c-8238-d6edf153bced admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f21b1fea950d48ef9a55cded8ed172a6'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:38.861778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6a51d2e7-e8f3-4e5c-8238-d6edf153bced admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f21b1fea950d48ef9a55cded8ed172a6'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:38.862675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6a51d2e7-e8f3-4e5c-8238-d6edf153bced admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f21b1fea950d48ef9a55cded8ed172a6'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:38.870585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 992/2019] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:38 2022] DELETE /identity/v3/users/f21b1fea950d48ef9a55cded8ed172a6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:39.081116 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746da9c-e03f-4f33-86a8-90eca539b570 service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:39.095359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746da9c-e03f-4f33-86a8-90eca539b570 service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:39.160718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba8367-1508-4092-8a99-3977923181f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:39.161362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba8367-1508-4092-8a99-3977923181f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:39.161639 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba8367-1508-4092-8a99-3977923181f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:39.161914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba8367-1508-4092-8a99-3977923181f7 admin admin] PATH_INFO: `/v3/projects/130966aabcc04fc98c9f5139f826e78f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:39.163048 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba8367-1508-4092-8a99-3977923181f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=130966aabcc04fc98c9f5139f826e78f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:39.165010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba8367-1508-4092-8a99-3977923181f7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:39.182658 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:39.217975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-72ba8367-1508-4092-8a99-3977923181f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '130966aabcc04fc98c9f5139f826e78f', 'request_id': 'req-72ba8367-1508-4092-8a99-3977923181f7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:39.224382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-72ba8367-1508-4092-8a99-3977923181f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '130966aabcc04fc98c9f5139f826e78f', 'request_id': 'req-72ba8367-1508-4092-8a99-3977923181f7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:39.233140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-72ba8367-1508-4092-8a99-3977923181f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '130966aabcc04fc98c9f5139f826e78f', 'request_id': 'req-72ba8367-1508-4092-8a99-3977923181f7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:39.249069 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-72ba8367-1508-4092-8a99-3977923181f7 admin admin] The token cache is being invalidate because project 130966aabcc04fc98c9f5139f826e78f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:39.249357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-72ba8367-1508-4092-8a99-3977923181f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:39.249982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-72ba8367-1508-4092-8a99-3977923181f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:39.251492 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1028/2020] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:39 2022] DELETE /identity/v3/projects/130966aabcc04fc98c9f5139f826e78f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:39.262399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb630fcf-705f-4ef5-988f-acb8fdb5a91b None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:39.262900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb630fcf-705f-4ef5-988f-acb8fdb5a91b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:39.262900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb630fcf-705f-4ef5-988f-acb8fdb5a91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:39.283845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-cb630fcf-705f-4ef5-988f-acb8fdb5a91b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:39.285176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-cb630fcf-705f-4ef5-988f-acb8fdb5a91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:39.351536 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 993/2021] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:39.360765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c1ee2-949e-4fc0-85f6-6f1d904046ce None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:39.361099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c1ee2-949e-4fc0-85f6-6f1d904046ce None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:39.361448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c1ee2-949e-4fc0-85f6-6f1d904046ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:39.385387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-af1c1ee2-949e-4fc0-85f6-6f1d904046ce None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:39.386803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-af1c1ee2-949e-4fc0-85f6-6f1d904046ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:39.395940 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1029/2022] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:37:39.402180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb630fcf-705f-4ef5-988f-acb8fdb5a91b None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:39.416006 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb630fcf-705f-4ef5-988f-acb8fdb5a91b None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:39.423003 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e5e87-2278-40f5-a9bb-8c3d4b6fdfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:39.423633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e5e87-2278-40f5-a9bb-8c3d4b6fdfcd admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:39.423942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e5e87-2278-40f5-a9bb-8c3d4b6fdfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:39.424206 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e5e87-2278-40f5-a9bb-8c3d4b6fdfcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:39.425017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e5e87-2278-40f5-a9bb-8c3d4b6fdfcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:37:39.425309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e5e87-2278-40f5-a9bb-8c3d4b6fdfcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:39.427444 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e5e87-2278-40f5-a9bb-8c3d4b6fdfcd admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:39.441070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 994/2023] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:37:39.445951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c1ee2-949e-4fc0-85f6-6f1d904046ce None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:39.455984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c1ee2-949e-4fc0-85f6-6f1d904046ce None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:39.461062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb128b3-a8af-4009-bc68-e4451d32f356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:39.461656 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb128b3-a8af-4009-bc68-e4451d32f356 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:39.461934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb128b3-a8af-4009-bc68-e4451d32f356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:39.462218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb128b3-a8af-4009-bc68-e4451d32f356 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:39.462964 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb128b3-a8af-4009-bc68-e4451d32f356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:39.465005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb128b3-a8af-4009-bc68-e4451d32f356 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:39.494612 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1030/2024] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:39 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:39.499483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e5e87-2278-40f5-a9bb-8c3d4b6fdfcd admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:39.515085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e5e87-2278-40f5-a9bb-8c3d4b6fdfcd admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:39.588797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4100be1-f0f5-406c-a037-6745697660fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:39.589750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4100be1-f0f5-406c-a037-6745697660fc admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:39.590031 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4100be1-f0f5-406c-a037-6745697660fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:39.590306 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4100be1-f0f5-406c-a037-6745697660fc admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:39.591027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4100be1-f0f5-406c-a037-6745697660fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:39.593027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4100be1-f0f5-406c-a037-6745697660fc admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:39.605050 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:39.621197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 995/2025] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:39 2022] POST /identity/v3/users => generated 302 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:37:39.626042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb128b3-a8af-4009-bc68-e4451d32f356 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:39.637330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb128b3-a8af-4009-bc68-e4451d32f356 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:39.642921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866787d-9182-4d27-9ec5-661a6bff51a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:39.643661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866787d-9182-4d27-9ec5-661a6bff51a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:39.644052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866787d-9182-4d27-9ec5-661a6bff51a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:39.644390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866787d-9182-4d27-9ec5-661a6bff51a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:39.645139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866787d-9182-4d27-9ec5-661a6bff51a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:39.646883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866787d-9182-4d27-9ec5-661a6bff51a3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:39.658105 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1031/2026] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:39.662308 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4100be1-f0f5-406c-a037-6745697660fc admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:39.673930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4100be1-f0f5-406c-a037-6745697660fc admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:39.679363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cf286-400f-4cc4-91af-b5eeb0770b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:39.680058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cf286-400f-4cc4-91af-b5eeb0770b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:39.680324 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cf286-400f-4cc4-91af-b5eeb0770b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:39.680592 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cf286-400f-4cc4-91af-b5eeb0770b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:39.681625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507cf286-400f-4cc4-91af-b5eeb0770b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:39.684095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507cf286-400f-4cc4-91af-b5eeb0770b18 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:39.700201 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 996/2027] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:39.705700 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866787d-9182-4d27-9ec5-661a6bff51a3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:39.719125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866787d-9182-4d27-9ec5-661a6bff51a3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:39.726262 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc70b70-b896-4720-909d-d1e9e32aea5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:39.726930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc70b70-b896-4720-909d-d1e9e32aea5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:39.727204 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc70b70-b896-4720-909d-d1e9e32aea5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:39.727473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc70b70-b896-4720-909d-d1e9e32aea5a admin admin] PATH_INFO: `/v3/projects/c3835ca60c464ab09dec6ba1da2bbe21/users/d3e887f7a0fd4b87ad22033d35bc3055/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:39.750713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc70b70-b896-4720-909d-d1e9e32aea5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3835ca60c464ab09dec6ba1da2bbe21, user_id=d3e887f7a0fd4b87ad22033d35bc3055, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:39.754251 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc70b70-b896-4720-909d-d1e9e32aea5a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:39.774449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1032/2028] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:39 2022] PUT /identity/v3/projects/c3835ca60c464ab09dec6ba1da2bbe21/users/d3e887f7a0fd4b87ad22033d35bc3055/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:39.786131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac763d0-9fd9-4cc1-a714-e9663fc7c48a None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:39.786486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac763d0-9fd9-4cc1-a714-e9663fc7c48a None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:39.786753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac763d0-9fd9-4cc1-a714-e9663fc7c48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:39.820774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6ac763d0-9fd9-4cc1-a714-e9663fc7c48a None None] MFA Rules not processed for user `d3e887f7a0fd4b87ad22033d35bc3055`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:39.822433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6ac763d0-9fd9-4cc1-a714-e9663fc7c48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:39.908113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 997/2029] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:39 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:39.913101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc70b70-b896-4720-909d-d1e9e32aea5a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:39.982254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc70b70-b896-4720-909d-d1e9e32aea5a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:39.987062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577daba-d519-46ca-962a-33d19e9bad7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:39.987671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577daba-d519-46ca-962a-33d19e9bad7c admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:39.987979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577daba-d519-46ca-962a-33d19e9bad7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:39.988249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577daba-d519-46ca-962a-33d19e9bad7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:39.988895 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3577daba-d519-46ca-962a-33d19e9bad7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:39.990809 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3577daba-d519-46ca-962a-33d19e9bad7c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:40.022508 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1033/2030] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:39 2022] POST /identity/v3/projects => generated 405 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:40.027108 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac763d0-9fd9-4cc1-a714-e9663fc7c48a None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:40.039411 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac763d0-9fd9-4cc1-a714-e9663fc7c48a None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:40.112384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a2990-df18-4300-a99c-ea0e07525caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:40.113733 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a2990-df18-4300-a99c-ea0e07525caf admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:40.114451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a2990-df18-4300-a99c-ea0e07525caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:40.115240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a2990-df18-4300-a99c-ea0e07525caf admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:40.116963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a2990-df18-4300-a99c-ea0e07525caf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:40.120491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a2990-df18-4300-a99c-ea0e07525caf admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:40.149622 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 998/2031] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:40 2022] POST /identity/v3/users => generated 309 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:40.151215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577daba-d519-46ca-962a-33d19e9bad7c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:40.163580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577daba-d519-46ca-962a-33d19e9bad7c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:40.170182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792fd84-00e1-453b-be3a-c3812ca04cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:40.170821 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792fd84-00e1-453b-be3a-c3812ca04cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:40.170987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792fd84-00e1-453b-be3a-c3812ca04cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:40.171197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792fd84-00e1-453b-be3a-c3812ca04cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:40.171877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792fd84-00e1-453b-be3a-c3812ca04cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:40.173675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792fd84-00e1-453b-be3a-c3812ca04cc7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:40.190979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1034/2032] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:40.193682 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a2990-df18-4300-a99c-ea0e07525caf admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:40.204750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a2990-df18-4300-a99c-ea0e07525caf admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:40.209526 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cd170-6363-4f37-b622-bc84a76b680f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:40.210255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cd170-6363-4f37-b622-bc84a76b680f admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:40.210600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cd170-6363-4f37-b622-bc84a76b680f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:40.210933 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cd170-6363-4f37-b622-bc84a76b680f admin admin] PATH_INFO: `/v3/projects/368b1defd5f446ada2b517f44147ad85/users/326aa5c59c914b678cb04b8697a66d70/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:40.232456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cd170-6363-4f37-b622-bc84a76b680f admin admin] RBAC: Authorizing `identity:create_grant(project_id=368b1defd5f446ada2b517f44147ad85, user_id=326aa5c59c914b678cb04b8697a66d70, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:40.235207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cd170-6363-4f37-b622-bc84a76b680f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:40.250265 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 999/2033] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:40 2022] PUT /identity/v3/projects/368b1defd5f446ada2b517f44147ad85/users/326aa5c59c914b678cb04b8697a66d70/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:40.254541 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792fd84-00e1-453b-be3a-c3812ca04cc7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:40.265516 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792fd84-00e1-453b-be3a-c3812ca04cc7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:40.277671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:40.329289 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c9341-b61d-4eba-a904-c9f34158e179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:40.329819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c9341-b61d-4eba-a904-c9f34158e179 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:40.330096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c9341-b61d-4eba-a904-c9f34158e179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:40.330383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c9341-b61d-4eba-a904-c9f34158e179 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:40.331006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c9341-b61d-4eba-a904-c9f34158e179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:40.332793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c9341-b61d-4eba-a904-c9f34158e179 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:40.348384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1035/2034] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:40.348812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cd170-6363-4f37-b622-bc84a76b680f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:40.361329 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cd170-6363-4f37-b622-bc84a76b680f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:40.366619 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e4d68-bc0e-4040-9ee2-1da531297150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:40.367397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4d68-bc0e-4040-9ee2-1da531297150 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:40.367774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4d68-bc0e-4040-9ee2-1da531297150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:40.368178 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4d68-bc0e-4040-9ee2-1da531297150 admin admin] PATH_INFO: `/v3/projects/368b1defd5f446ada2b517f44147ad85/users/326aa5c59c914b678cb04b8697a66d70/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:40.371398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e4d68-bc0e-4040-9ee2-1da531297150 admin admin] RBAC: Authorizing `identity:create_grant(project_id=368b1defd5f446ada2b517f44147ad85, user_id=326aa5c59c914b678cb04b8697a66d70, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:40.380978 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e4d68-bc0e-4040-9ee2-1da531297150 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:40.393010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1000/2035] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:40 2022] PUT /identity/v3/projects/368b1defd5f446ada2b517f44147ad85/users/326aa5c59c914b678cb04b8697a66d70/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:40.407747 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcc636-5109-4ab0-8a16-6b0407fadb3a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:40.408065 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcc636-5109-4ab0-8a16-6b0407fadb3a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:40.408337 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcc636-5109-4ab0-8a16-6b0407fadb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:40.439728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-45bcc636-5109-4ab0-8a16-6b0407fadb3a None None] MFA Rules not processed for user `326aa5c59c914b678cb04b8697a66d70`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:40.441361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-45bcc636-5109-4ab0-8a16-6b0407fadb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:40.537719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1036/2036] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:40 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:40.547024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e4d68-bc0e-4040-9ee2-1da531297150 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:40.549803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f08e4d68-bc0e-4040-9ee2-1da531297150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:40.580676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:40.614582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e4d68-bc0e-4040-9ee2-1da531297150 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:40.618527 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3956a6-2681-4d5a-aa19-595790c37240 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:40.619338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3956a6-2681-4d5a-aa19-595790c37240 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:40.619613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3956a6-2681-4d5a-aa19-595790c37240 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:40.619882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3956a6-2681-4d5a-aa19-595790c37240 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:40.629513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3956a6-2681-4d5a-aa19-595790c37240 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:40.630959 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3956a6-2681-4d5a-aa19-595790c37240 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:40.642315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1001/2037] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:40 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:40.836043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcc636-5109-4ab0-8a16-6b0407fadb3a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:40.913245 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcc636-5109-4ab0-8a16-6b0407fadb3a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:40.920540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da24b7-1895-4e12-8864-4a356bbb8748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:40.921224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da24b7-1895-4e12-8864-4a356bbb8748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:40.921498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da24b7-1895-4e12-8864-4a356bbb8748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:40.921734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da24b7-1895-4e12-8864-4a356bbb8748 admin admin] PATH_INFO: `/v3/users/d3e887f7a0fd4b87ad22033d35bc3055` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:40.923047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da24b7-1895-4e12-8864-4a356bbb8748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3e887f7a0fd4b87ad22033d35bc3055)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:40.925008 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da24b7-1895-4e12-8864-4a356bbb8748 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:40.978949 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:40.987599 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-58da24b7-1895-4e12-8864-4a356bbb8748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3e887f7a0fd4b87ad22033d35bc3055'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:40.996921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-58da24b7-1895-4e12-8864-4a356bbb8748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3e887f7a0fd4b87ad22033d35bc3055'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:40.997680 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-58da24b7-1895-4e12-8864-4a356bbb8748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3e887f7a0fd4b87ad22033d35bc3055'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:41.012799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-58da24b7-1895-4e12-8864-4a356bbb8748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3e887f7a0fd4b87ad22033d35bc3055'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:41.020246 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-58da24b7-1895-4e12-8864-4a356bbb8748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3e887f7a0fd4b87ad22033d35bc3055'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:41.026646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1037/2038] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:40 2022] DELETE /identity/v3/users/d3e887f7a0fd4b87ad22033d35bc3055 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:41.032127 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8b91-a13c-4bc8-8fe8-e0bf3ce3e090 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:41.032565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8b91-a13c-4bc8-8fe8-e0bf3ce3e090 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:41.032936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8b91-a13c-4bc8-8fe8-e0bf3ce3e090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:41.053451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-d65a8b91-a13c-4bc8-8fe8-e0bf3ce3e090 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:41.054735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d65a8b91-a13c-4bc8-8fe8-e0bf3ce3e090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:41.137300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1002/2039] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:37:41.147221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da24b7-1895-4e12-8864-4a356bbb8748 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:41.148701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-58da24b7-1895-4e12-8864-4a356bbb8748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:41.206786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da24b7-1895-4e12-8864-4a356bbb8748 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:41.210659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf1559-777a-4be6-b727-6f5fd428f438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:41.211257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf1559-777a-4be6-b727-6f5fd428f438 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:41.211525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf1559-777a-4be6-b727-6f5fd428f438 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:41.211777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf1559-777a-4be6-b727-6f5fd428f438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:41.221420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf1559-777a-4be6-b727-6f5fd428f438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:41.222828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf1559-777a-4be6-b727-6f5fd428f438 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:41.234157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1038/2040] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:41.578574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a8b91-a13c-4bc8-8fe8-e0bf3ce3e090 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:41.580268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d65a8b91-a13c-4bc8-8fe8-e0bf3ce3e090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:41.618879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a8b91-a13c-4bc8-8fe8-e0bf3ce3e090 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:41.624393 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:41.625047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:41.625325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:41.625599 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] PATH_INFO: `/v3/projects/c3835ca60c464ab09dec6ba1da2bbe21` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:41.627313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3835ca60c464ab09dec6ba1da2bbe21)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:41.629805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:41.687661 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3835ca60c464ab09dec6ba1da2bbe21', 'request_id': 'req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:41.699147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3835ca60c464ab09dec6ba1da2bbe21', 'request_id': 'req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:41.713221 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3835ca60c464ab09dec6ba1da2bbe21', 'request_id': 'req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:41.726995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] The token cache is being invalidate because project c3835ca60c464ab09dec6ba1da2bbe21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:41.727274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:41.728084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:41.730059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1003/2041] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:41 2022] DELETE /identity/v3/projects/c3835ca60c464ab09dec6ba1da2bbe21 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:41.734231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf1559-777a-4be6-b727-6f5fd428f438 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:41.745213 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf1559-777a-4be6-b727-6f5fd428f438 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:41.831794 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9b203-6807-48c6-b992-d14d73019a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:41.832765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9b203-6807-48c6-b992-d14d73019a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:41.833169 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9b203-6807-48c6-b992-d14d73019a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:41.833445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9b203-6807-48c6-b992-d14d73019a5e admin admin] PATH_INFO: `/v3/users/326aa5c59c914b678cb04b8697a66d70` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:41.834808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9b203-6807-48c6-b992-d14d73019a5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=326aa5c59c914b678cb04b8697a66d70)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:41.836801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9b203-6807-48c6-b992-d14d73019a5e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:41.891828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c0e9b203-6807-48c6-b992-d14d73019a5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326aa5c59c914b678cb04b8697a66d70'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:41.901372 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c0e9b203-6807-48c6-b992-d14d73019a5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326aa5c59c914b678cb04b8697a66d70'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:41.902289 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c0e9b203-6807-48c6-b992-d14d73019a5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326aa5c59c914b678cb04b8697a66d70'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:41.912755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:41.925065 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c0e9b203-6807-48c6-b992-d14d73019a5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326aa5c59c914b678cb04b8697a66d70'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:41.933829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c0e9b203-6807-48c6-b992-d14d73019a5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326aa5c59c914b678cb04b8697a66d70'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:41.940149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1039/2042] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:41 2022] DELETE /identity/v3/users/326aa5c59c914b678cb04b8697a66d70 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:37:42.277206 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:42.278822 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:42.294095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35a3ee-4f3d-48e7-ac12-4f391cba219f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:42.349552 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:42.365722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:42.366389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:42.366660 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:42.366937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee admin admin] PATH_INFO: `/v3/projects/368b1defd5f446ada2b517f44147ad85` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:42.368175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=368b1defd5f446ada2b517f44147ad85)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:42.370385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:42.433258 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '368b1defd5f446ada2b517f44147ad85', 'request_id': 'req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:42.444831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '368b1defd5f446ada2b517f44147ad85', 'request_id': 'req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:42.468700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '368b1defd5f446ada2b517f44147ad85', 'request_id': 'req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:42.489652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee admin admin] The token cache is being invalidate because project 368b1defd5f446ada2b517f44147ad85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:42.489935 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:42.490598 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-be3c7bb2-1408-479d-9b8e-d340e9c2c6ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:42.492161 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1004/2043] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:42 2022] DELETE /identity/v3/projects/368b1defd5f446ada2b517f44147ad85 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:37:42.497975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be40cb6-06aa-47f0-8984-39107627195b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:42.498248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be40cb6-06aa-47f0-8984-39107627195b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:42.498517 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be40cb6-06aa-47f0-8984-39107627195b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:42.518182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8be40cb6-06aa-47f0-8984-39107627195b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:42.519389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8be40cb6-06aa-47f0-8984-39107627195b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:42.596140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1040/2044] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:42.597784 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081aa1f-c38a-498a-8d71-8057b956eb62 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:42.598509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081aa1f-c38a-498a-8d71-8057b956eb62 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:42.598509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081aa1f-c38a-498a-8d71-8057b956eb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:42.620798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-5081aa1f-c38a-498a-8d71-8057b956eb62 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:42.622356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5081aa1f-c38a-498a-8d71-8057b956eb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:42.631789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1005/2045] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:37:42.636165 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be40cb6-06aa-47f0-8984-39107627195b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:42.646676 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be40cb6-06aa-47f0-8984-39107627195b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:42.653176 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbf6c4-6fec-4371-9636-d4dd24b22364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:42.653948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf6c4-6fec-4371-9636-d4dd24b22364 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:42.654313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf6c4-6fec-4371-9636-d4dd24b22364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:42.654643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbf6c4-6fec-4371-9636-d4dd24b22364 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:42.655362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbf6c4-6fec-4371-9636-d4dd24b22364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:37:42.655727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbf6c4-6fec-4371-9636-d4dd24b22364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:42.657449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dbf6c4-6fec-4371-9636-d4dd24b22364 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:42.671083 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1041/2046] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:37:42.675472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081aa1f-c38a-498a-8d71-8057b956eb62 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:42.685220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081aa1f-c38a-498a-8d71-8057b956eb62 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:42.689964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cf4b3-1e87-4d47-9298-61a1dd6896fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:42.690567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cf4b3-1e87-4d47-9298-61a1dd6896fe admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:42.690838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cf4b3-1e87-4d47-9298-61a1dd6896fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:42.691103 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cf4b3-1e87-4d47-9298-61a1dd6896fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:42.691772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cf4b3-1e87-4d47-9298-61a1dd6896fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:42.693714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cf4b3-1e87-4d47-9298-61a1dd6896fe admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:42.741142 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1006/2047] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:42 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:37:42.746355 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbf6c4-6fec-4371-9636-d4dd24b22364 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:42.758319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbf6c4-6fec-4371-9636-d4dd24b22364 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:42.831327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670452fb-10e5-499c-b390-2d3be13d16e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:42.831916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670452fb-10e5-499c-b390-2d3be13d16e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:42.832208 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670452fb-10e5-499c-b390-2d3be13d16e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:42.832472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670452fb-10e5-499c-b390-2d3be13d16e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:42.833146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670452fb-10e5-499c-b390-2d3be13d16e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:42.835112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670452fb-10e5-499c-b390-2d3be13d16e8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:42.865986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1042/2048] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:42 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:42.868710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cf4b3-1e87-4d47-9298-61a1dd6896fe admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:42.881353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cf4b3-1e87-4d47-9298-61a1dd6896fe admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:42.886425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ac3e7-45b7-4edb-9ead-f1d19f06a5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:42.887037 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ac3e7-45b7-4edb-9ead-f1d19f06a5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:42.887307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ac3e7-45b7-4edb-9ead-f1d19f06a5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:42.887573 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ac3e7-45b7-4edb-9ead-f1d19f06a5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:42.888371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ac3e7-45b7-4edb-9ead-f1d19f06a5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:42.889958 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ac3e7-45b7-4edb-9ead-f1d19f06a5e6 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:42.900532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1007/2049] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:42.905037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670452fb-10e5-499c-b390-2d3be13d16e8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:42.915867 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670452fb-10e5-499c-b390-2d3be13d16e8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:42.920685 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34491ceb-c715-4c86-b039-37491eec8461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:42.921282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34491ceb-c715-4c86-b039-37491eec8461 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:42.921555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34491ceb-c715-4c86-b039-37491eec8461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:42.921819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34491ceb-c715-4c86-b039-37491eec8461 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:42.922450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34491ceb-c715-4c86-b039-37491eec8461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:42.924071 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34491ceb-c715-4c86-b039-37491eec8461 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:42.935077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1043/2050] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:42.939180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ac3e7-45b7-4edb-9ead-f1d19f06a5e6 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:42.949515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ac3e7-45b7-4edb-9ead-f1d19f06a5e6 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:42.954989 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa144c63-0929-4708-99c7-aad04ac0aab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:42.955641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa144c63-0929-4708-99c7-aad04ac0aab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:42.955916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa144c63-0929-4708-99c7-aad04ac0aab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:42.956199 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa144c63-0929-4708-99c7-aad04ac0aab2 admin admin] PATH_INFO: `/v3/projects/237b1c318adb4cf4a5f4b3685af554e7/users/bdb3c71e5ef3418cae4045b6dab1340f/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:42.971874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa144c63-0929-4708-99c7-aad04ac0aab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=237b1c318adb4cf4a5f4b3685af554e7, user_id=bdb3c71e5ef3418cae4045b6dab1340f, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:42.974738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa144c63-0929-4708-99c7-aad04ac0aab2 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:42.990030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1008/2051] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:42 2022] PUT /identity/v3/projects/237b1c318adb4cf4a5f4b3685af554e7/users/bdb3c71e5ef3418cae4045b6dab1340f/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:43.001640 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37082dd-abe6-47e6-b70d-5404ca4855a9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:43.002002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37082dd-abe6-47e6-b70d-5404ca4855a9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:43.002401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37082dd-abe6-47e6-b70d-5404ca4855a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:43.044955 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e37082dd-abe6-47e6-b70d-5404ca4855a9 None None] MFA Rules not processed for user `bdb3c71e5ef3418cae4045b6dab1340f`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:43.046184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e37082dd-abe6-47e6-b70d-5404ca4855a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:43.102282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:43.148649 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1044/2052] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:43.153803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa144c63-0929-4708-99c7-aad04ac0aab2 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:43.239655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa144c63-0929-4708-99c7-aad04ac0aab2 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:43.246815 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d12a3-71c7-439e-b51e-3c60b2749e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:43.247522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d12a3-71c7-439e-b51e-3c60b2749e4f admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:43.247849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d12a3-71c7-439e-b51e-3c60b2749e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:43.248198 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d12a3-71c7-439e-b51e-3c60b2749e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:43.248919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d12a3-71c7-439e-b51e-3c60b2749e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:43.252925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d12a3-71c7-439e-b51e-3c60b2749e4f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:43.275436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:43.283927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1009/2053] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:43 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:43.287641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37082dd-abe6-47e6-b70d-5404ca4855a9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:43.299376 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37082dd-abe6-47e6-b70d-5404ca4855a9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:43.361922 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d2149-6736-43aa-abb0-aba2d868530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:43.362682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d2149-6736-43aa-abb0-aba2d868530e admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:43.362956 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d2149-6736-43aa-abb0-aba2d868530e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:43.363247 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d2149-6736-43aa-abb0-aba2d868530e admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:43.363963 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d2149-6736-43aa-abb0-aba2d868530e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:43.365996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d2149-6736-43aa-abb0-aba2d868530e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:43.403358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1045/2054] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:43 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:43.407608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d12a3-71c7-439e-b51e-3c60b2749e4f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:43.420400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d12a3-71c7-439e-b51e-3c60b2749e4f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:43.432461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203e7d0-4010-41ca-abff-3a1f518d0cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:43.433284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203e7d0-4010-41ca-abff-3a1f518d0cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:43.433690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203e7d0-4010-41ca-abff-3a1f518d0cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:43.434040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203e7d0-4010-41ca-abff-3a1f518d0cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:43.434996 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203e7d0-4010-41ca-abff-3a1f518d0cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:43.437090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203e7d0-4010-41ca-abff-3a1f518d0cc1 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:43.449937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1010/2055] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:43.454934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d2149-6736-43aa-abb0-aba2d868530e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:43.466020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d2149-6736-43aa-abb0-aba2d868530e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:43.472301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e5732-21fe-47da-a581-14c8411ee372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:43.472948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e5732-21fe-47da-a581-14c8411ee372 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:43.473249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e5732-21fe-47da-a581-14c8411ee372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:43.473512 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e5732-21fe-47da-a581-14c8411ee372 admin admin] PATH_INFO: `/v3/projects/930307745e1948d1a0da371aa4a1276d/users/edde78efe3b1483b9c90843801662ef1/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:43.490409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e5732-21fe-47da-a581-14c8411ee372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=930307745e1948d1a0da371aa4a1276d, user_id=edde78efe3b1483b9c90843801662ef1, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:43.493522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e5732-21fe-47da-a581-14c8411ee372 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:43.511136 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1046/2056] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:43 2022] PUT /identity/v3/projects/930307745e1948d1a0da371aa4a1276d/users/edde78efe3b1483b9c90843801662ef1/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:43.515930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203e7d0-4010-41ca-abff-3a1f518d0cc1 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:43.531165 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203e7d0-4010-41ca-abff-3a1f518d0cc1 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:43.591724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e31f9-edfc-4b24-b795-8e9e78066250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:43.592413 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e31f9-edfc-4b24-b795-8e9e78066250 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:43.592714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e31f9-edfc-4b24-b795-8e9e78066250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:43.592998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e31f9-edfc-4b24-b795-8e9e78066250 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:43.593669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e31f9-edfc-4b24-b795-8e9e78066250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:43.595382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e31f9-edfc-4b24-b795-8e9e78066250 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:43.606262 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1011/2057] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:43.610898 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e5732-21fe-47da-a581-14c8411ee372 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:43.623000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e5732-21fe-47da-a581-14c8411ee372 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:43.628344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4735f-4585-4dfe-a9c5-28b1330af4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:43.629007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4735f-4585-4dfe-a9c5-28b1330af4e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:43.629315 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4735f-4585-4dfe-a9c5-28b1330af4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:43.629618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4735f-4585-4dfe-a9c5-28b1330af4e0 admin admin] PATH_INFO: `/v3/projects/930307745e1948d1a0da371aa4a1276d/users/edde78efe3b1483b9c90843801662ef1/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:43.631624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4735f-4585-4dfe-a9c5-28b1330af4e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=930307745e1948d1a0da371aa4a1276d, user_id=edde78efe3b1483b9c90843801662ef1, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:43.634674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4735f-4585-4dfe-a9c5-28b1330af4e0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:43.646905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1047/2058] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:43 2022] PUT /identity/v3/projects/930307745e1948d1a0da371aa4a1276d/users/edde78efe3b1483b9c90843801662ef1/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:43.659030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f27f0-0cbc-4367-a18a-4431a081caca None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:43.659322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f27f0-0cbc-4367-a18a-4431a081caca None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:43.659605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f27f0-0cbc-4367-a18a-4431a081caca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:43.704783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-345f27f0-0cbc-4367-a18a-4431a081caca None None] MFA Rules not processed for user `edde78efe3b1483b9c90843801662ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:43.706154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-345f27f0-0cbc-4367-a18a-4431a081caca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:43.784345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1012/2059] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:37:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:43.795801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4735f-4585-4dfe-a9c5-28b1330af4e0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:43.797750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-def4735f-4585-4dfe-a9c5-28b1330af4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:43.879144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4735f-4585-4dfe-a9c5-28b1330af4e0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:43.885946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbab65-1326-497f-a25f-b66b0a76a5ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:43.886934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbab65-1326-497f-a25f-b66b0a76a5ef service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:43.887338 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbab65-1326-497f-a25f-b66b0a76a5ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:43.887751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbab65-1326-497f-a25f-b66b0a76a5ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:43.903312 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbab65-1326-497f-a25f-b66b0a76a5ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:43.905802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbab65-1326-497f-a25f-b66b0a76a5ef service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:43.922331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1048/2060] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:44.052029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f27f0-0cbc-4367-a18a-4431a081caca None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:44.069097 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f27f0-0cbc-4367-a18a-4431a081caca None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:44.075062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac199ba9-1bcc-4d7a-a207-c5f20ba3d528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:44.076100 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac199ba9-1bcc-4d7a-a207-c5f20ba3d528 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:44.076544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac199ba9-1bcc-4d7a-a207-c5f20ba3d528 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:44.076987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac199ba9-1bcc-4d7a-a207-c5f20ba3d528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:44.165164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac199ba9-1bcc-4d7a-a207-c5f20ba3d528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:44.167039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac199ba9-1bcc-4d7a-a207-c5f20ba3d528 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:44.178603 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1013/2061] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:53.914325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:54.173691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=96 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:02.238874 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbab65-1326-497f-a25f-b66b0a76a5ef service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:02.242754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b7bbab65-1326-497f-a25f-b66b0a76a5ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:02.321817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbab65-1326-497f-a25f-b66b0a76a5ef service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:02.327086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f6331-0d98-4f0a-9869-10669bf0dcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:02.328148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f6331-0d98-4f0a-9869-10669bf0dcbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:02.328427 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f6331-0d98-4f0a-9869-10669bf0dcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:02.328769 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f6331-0d98-4f0a-9869-10669bf0dcbd admin admin] PATH_INFO: `/v3/users/22861dd3ba6e41c9b9d92122a8f31183` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:02.330148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f6331-0d98-4f0a-9869-10669bf0dcbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=22861dd3ba6e41c9b9d92122a8f31183)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:02.332177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f6331-0d98-4f0a-9869-10669bf0dcbd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:02.404155 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-980f6331-0d98-4f0a-9869-10669bf0dcbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22861dd3ba6e41c9b9d92122a8f31183'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:02.413802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-980f6331-0d98-4f0a-9869-10669bf0dcbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22861dd3ba6e41c9b9d92122a8f31183'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:02.414496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-980f6331-0d98-4f0a-9869-10669bf0dcbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22861dd3ba6e41c9b9d92122a8f31183'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:02.430589 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-980f6331-0d98-4f0a-9869-10669bf0dcbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22861dd3ba6e41c9b9d92122a8f31183'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:02.440450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-980f6331-0d98-4f0a-9869-10669bf0dcbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22861dd3ba6e41c9b9d92122a8f31183'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:02.448441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1049/2062] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:02 2022] DELETE /identity/v3/users/22861dd3ba6e41c9b9d92122a8f31183 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:02.456200 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7a5b4-5c27-4b93-a7e0-1dce9bf65343 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:02.456739 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7a5b4-5c27-4b93-a7e0-1dce9bf65343 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:02.456739 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7a5b4-5c27-4b93-a7e0-1dce9bf65343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:02.480411 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e2f7a5b4-5c27-4b93-a7e0-1dce9bf65343 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:02.481878 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e2f7a5b4-5c27-4b93-a7e0-1dce9bf65343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:02.589390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1014/2063] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:02.599623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f6331-0d98-4f0a-9869-10669bf0dcbd admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:02.602492 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-980f6331-0d98-4f0a-9869-10669bf0dcbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:02.722701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f6331-0d98-4f0a-9869-10669bf0dcbd admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:02.728498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f57f8b-eac7-486b-991f-fd49c1d94ba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:02.729199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f57f8b-eac7-486b-991f-fd49c1d94ba0 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:02.729627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f57f8b-eac7-486b-991f-fd49c1d94ba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:02.729964 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f57f8b-eac7-486b-991f-fd49c1d94ba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:02.742811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f57f8b-eac7-486b-991f-fd49c1d94ba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:02.744516 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f57f8b-eac7-486b-991f-fd49c1d94ba0 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:02.761391 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1050/2064] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:03.085472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7a5b4-5c27-4b93-a7e0-1dce9bf65343 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:03.086975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e2f7a5b4-5c27-4b93-a7e0-1dce9bf65343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:03.100280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7a5b4-5c27-4b93-a7e0-1dce9bf65343 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:03.106069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3482c-94a0-4350-952c-b876466f615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:03.106797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3482c-94a0-4350-952c-b876466f615a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:03.106993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3482c-94a0-4350-952c-b876466f615a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:03.107798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3482c-94a0-4350-952c-b876466f615a admin admin] PATH_INFO: `/v3/projects/c85bd65bebf14ec490166c4971f26bf4` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:03.108428 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3482c-94a0-4350-952c-b876466f615a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c85bd65bebf14ec490166c4971f26bf4)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:03.110338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3482c-94a0-4350-952c-b876466f615a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:03.184053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e4b3482c-94a0-4350-952c-b876466f615a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c85bd65bebf14ec490166c4971f26bf4', 'request_id': 'req-e4b3482c-94a0-4350-952c-b876466f615a'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:03.193266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e4b3482c-94a0-4350-952c-b876466f615a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c85bd65bebf14ec490166c4971f26bf4', 'request_id': 'req-e4b3482c-94a0-4350-952c-b876466f615a'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:03.200504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e4b3482c-94a0-4350-952c-b876466f615a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c85bd65bebf14ec490166c4971f26bf4', 'request_id': 'req-e4b3482c-94a0-4350-952c-b876466f615a'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:03.213243 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e4b3482c-94a0-4350-952c-b876466f615a admin admin] The token cache is being invalidate because project c85bd65bebf14ec490166c4971f26bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:03.213525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e4b3482c-94a0-4350-952c-b876466f615a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:03.214194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e4b3482c-94a0-4350-952c-b876466f615a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:03.215760 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1015/2065] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:03 2022] DELETE /identity/v3/projects/c85bd65bebf14ec490166c4971f26bf4 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:03.221167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc8b03-5e48-4273-9d01-76022b49157f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:03.221661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc8b03-5e48-4273-9d01-76022b49157f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:03.221935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc8b03-5e48-4273-9d01-76022b49157f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:03.247391 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bfdc8b03-5e48-4273-9d01-76022b49157f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:03.248721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bfdc8b03-5e48-4273-9d01-76022b49157f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:03.268356 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:03.323085 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1051/2066] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:03.331198 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99545b-c82e-470a-9648-f3aed7f483b0 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:03.331482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99545b-c82e-470a-9648-f3aed7f483b0 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:03.331752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99545b-c82e-470a-9648-f3aed7f483b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:03.352737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-da99545b-c82e-470a-9648-f3aed7f483b0 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:03.353954 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-da99545b-c82e-470a-9648-f3aed7f483b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:03.383691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1016/2067] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:03.392146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc8b03-5e48-4273-9d01-76022b49157f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:03.398712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc8b03-5e48-4273-9d01-76022b49157f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:03.405038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849d5f4-b741-45f1-b236-0c20d4d156dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:03.405976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849d5f4-b741-45f1-b236-0c20d4d156dd admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:03.406316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849d5f4-b741-45f1-b236-0c20d4d156dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:03.406701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849d5f4-b741-45f1-b236-0c20d4d156dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:03.407692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2849d5f4-b741-45f1-b236-0c20d4d156dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:38:03.408086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2849d5f4-b741-45f1-b236-0c20d4d156dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:03.409876 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2849d5f4-b741-45f1-b236-0c20d4d156dd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:03.424149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1052/2068] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:38:03.428698 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99545b-c82e-470a-9648-f3aed7f483b0 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:03.440336 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99545b-c82e-470a-9648-f3aed7f483b0 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:03.446410 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7c7da-8b46-4daf-96d0-51fc81ccbc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:03.447332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7c7da-8b46-4daf-96d0-51fc81ccbc17 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:03.448003 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7c7da-8b46-4daf-96d0-51fc81ccbc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:03.448422 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7c7da-8b46-4daf-96d0-51fc81ccbc17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:03.449412 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7c7da-8b46-4daf-96d0-51fc81ccbc17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:03.451482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7c7da-8b46-4daf-96d0-51fc81ccbc17 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:03.495572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1017/2069] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:03 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:03.500056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849d5f4-b741-45f1-b236-0c20d4d156dd admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:03.512143 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849d5f4-b741-45f1-b236-0c20d4d156dd admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:03.583416 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514e4c6-9de8-4b7b-a08d-11f4a755db00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:03.584122 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514e4c6-9de8-4b7b-a08d-11f4a755db00 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:03.584438 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514e4c6-9de8-4b7b-a08d-11f4a755db00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:03.584706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514e4c6-9de8-4b7b-a08d-11f4a755db00 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:03.586007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514e4c6-9de8-4b7b-a08d-11f4a755db00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:03.587966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514e4c6-9de8-4b7b-a08d-11f4a755db00 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:03.617400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1053/2070] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:03 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:38:03.619141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7c7da-8b46-4daf-96d0-51fc81ccbc17 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:03.630572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7c7da-8b46-4daf-96d0-51fc81ccbc17 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:03.636886 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40ace0-9acc-4f79-9120-bd6371da49cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:03.637735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40ace0-9acc-4f79-9120-bd6371da49cb admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:03.638032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40ace0-9acc-4f79-9120-bd6371da49cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:03.638279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40ace0-9acc-4f79-9120-bd6371da49cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:03.638935 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40ace0-9acc-4f79-9120-bd6371da49cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:03.640604 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40ace0-9acc-4f79-9120-bd6371da49cb admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:03.653298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1018/2071] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:03.658185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514e4c6-9de8-4b7b-a08d-11f4a755db00 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:03.672356 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514e4c6-9de8-4b7b-a08d-11f4a755db00 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:03.679939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3f530-ec19-4c67-a8e1-992c6b0d744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:03.680669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f530-ec19-4c67-a8e1-992c6b0d744a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:03.680943 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f530-ec19-4c67-a8e1-992c6b0d744a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:03.681206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f530-ec19-4c67-a8e1-992c6b0d744a admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:03.681860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3f530-ec19-4c67-a8e1-992c6b0d744a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:03.683582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3f530-ec19-4c67-a8e1-992c6b0d744a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:03.696245 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1054/2072] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:03.700751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40ace0-9acc-4f79-9120-bd6371da49cb admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:03.713245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40ace0-9acc-4f79-9120-bd6371da49cb admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:03.719038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b10b3-8d04-4e28-abbe-920b234cc2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:03.719748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b10b3-8d04-4e28-abbe-920b234cc2e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:03.720092 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b10b3-8d04-4e28-abbe-920b234cc2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:03.720361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b10b3-8d04-4e28-abbe-920b234cc2e4 admin admin] PATH_INFO: `/v3/projects/bf1b4c4e32ba4f8a8fdda48b930d55fa/users/91d55246530449759d27cd9995da780c/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:03.737977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b10b3-8d04-4e28-abbe-920b234cc2e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf1b4c4e32ba4f8a8fdda48b930d55fa, user_id=91d55246530449759d27cd9995da780c, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:03.740720 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b10b3-8d04-4e28-abbe-920b234cc2e4 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:03.757223 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1019/2073] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:03 2022] PUT /identity/v3/projects/bf1b4c4e32ba4f8a8fdda48b930d55fa/users/91d55246530449759d27cd9995da780c/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:03.771270 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd2e87-6302-41ef-872e-fc821c4c5c69 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:03.771554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd2e87-6302-41ef-872e-fc821c4c5c69 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:03.771817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd2e87-6302-41ef-872e-fc821c4c5c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:03.810078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c4dd2e87-6302-41ef-872e-fc821c4c5c69 None None] MFA Rules not processed for user `91d55246530449759d27cd9995da780c`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:03.811496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c4dd2e87-6302-41ef-872e-fc821c4c5c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:03.904889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1055/2074] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:03.917195 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b10b3-8d04-4e28-abbe-920b234cc2e4 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:03.919171 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8b7b10b3-8d04-4e28-abbe-920b234cc2e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:03.947703 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:03.990917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b10b3-8d04-4e28-abbe-920b234cc2e4 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:03.995071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495f7d3-d0da-468c-ad78-84c37df1f583 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:03.995699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495f7d3-d0da-468c-ad78-84c37df1f583 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:03.996020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495f7d3-d0da-468c-ad78-84c37df1f583 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:03.996442 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495f7d3-d0da-468c-ad78-84c37df1f583 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:04.007126 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495f7d3-d0da-468c-ad78-84c37df1f583 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:04.008348 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495f7d3-d0da-468c-ad78-84c37df1f583 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:04.020440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1020/2075] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:04.051383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd2e87-6302-41ef-872e-fc821c4c5c69 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:04.078884 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:04.149895 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd2e87-6302-41ef-872e-fc821c4c5c69 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:04.155166 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01e628-f4d8-4ecd-9c06-a45996562ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:04.155967 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01e628-f4d8-4ecd-9c06-a45996562ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:04.156344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01e628-f4d8-4ecd-9c06-a45996562ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:04.156713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01e628-f4d8-4ecd-9c06-a45996562ddd admin admin] PATH_INFO: `/v3/users/91d55246530449759d27cd9995da780c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:04.157974 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01e628-f4d8-4ecd-9c06-a45996562ddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=91d55246530449759d27cd9995da780c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:04.169371 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01e628-f4d8-4ecd-9c06-a45996562ddd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:04.236347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4b01e628-f4d8-4ecd-9c06-a45996562ddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91d55246530449759d27cd9995da780c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:04.245862 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4b01e628-f4d8-4ecd-9c06-a45996562ddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91d55246530449759d27cd9995da780c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:04.246499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4b01e628-f4d8-4ecd-9c06-a45996562ddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91d55246530449759d27cd9995da780c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:04.260965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4b01e628-f4d8-4ecd-9c06-a45996562ddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91d55246530449759d27cd9995da780c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:04.268401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4b01e628-f4d8-4ecd-9c06-a45996562ddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91d55246530449759d27cd9995da780c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:04.275104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1056/2076] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:04 2022] DELETE /identity/v3/users/91d55246530449759d27cd9995da780c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:04.279600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ca3f0-3887-41b3-a250-3e2f4a314de7 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:04.279932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ca3f0-3887-41b3-a250-3e2f4a314de7 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:04.280216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ca3f0-3887-41b3-a250-3e2f4a314de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:04.302097 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-3d9ca3f0-3887-41b3-a250-3e2f4a314de7 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:04.303592 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3d9ca3f0-3887-41b3-a250-3e2f4a314de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:04.374268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1021/2077] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:04.385542 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01e628-f4d8-4ecd-9c06-a45996562ddd admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:04.387418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4b01e628-f4d8-4ecd-9c06-a45996562ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:04.466755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01e628-f4d8-4ecd-9c06-a45996562ddd admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:04.470702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f784c95-2237-4301-827e-aad0cbf42b6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:04.471331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f784c95-2237-4301-827e-aad0cbf42b6e service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:04.471602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f784c95-2237-4301-827e-aad0cbf42b6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:04.472130 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f784c95-2237-4301-827e-aad0cbf42b6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:04.482237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f784c95-2237-4301-827e-aad0cbf42b6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:04.483751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f784c95-2237-4301-827e-aad0cbf42b6e service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:04.498139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1057/2078] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:04.805681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ca3f0-3887-41b3-a250-3e2f4a314de7 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:04.807496 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3d9ca3f0-3887-41b3-a250-3e2f4a314de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:04.822272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ca3f0-3887-41b3-a250-3e2f4a314de7 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:04.829220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316baf6-eb07-4d03-b82f-b6a28f99c43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:04.830055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316baf6-eb07-4d03-b82f-b6a28f99c43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:04.830335 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316baf6-eb07-4d03-b82f-b6a28f99c43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:04.830696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316baf6-eb07-4d03-b82f-b6a28f99c43f admin admin] PATH_INFO: `/v3/projects/bf1b4c4e32ba4f8a8fdda48b930d55fa` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:04.831835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316baf6-eb07-4d03-b82f-b6a28f99c43f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf1b4c4e32ba4f8a8fdda48b930d55fa)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:04.833860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316baf6-eb07-4d03-b82f-b6a28f99c43f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:04.893684 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f316baf6-eb07-4d03-b82f-b6a28f99c43f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf1b4c4e32ba4f8a8fdda48b930d55fa', 'request_id': 'req-f316baf6-eb07-4d03-b82f-b6a28f99c43f'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:04.902799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f316baf6-eb07-4d03-b82f-b6a28f99c43f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf1b4c4e32ba4f8a8fdda48b930d55fa', 'request_id': 'req-f316baf6-eb07-4d03-b82f-b6a28f99c43f'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:04.908824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f316baf6-eb07-4d03-b82f-b6a28f99c43f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf1b4c4e32ba4f8a8fdda48b930d55fa', 'request_id': 'req-f316baf6-eb07-4d03-b82f-b6a28f99c43f'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:04.920644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f316baf6-eb07-4d03-b82f-b6a28f99c43f admin admin] The token cache is being invalidate because project bf1b4c4e32ba4f8a8fdda48b930d55fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:04.920929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f316baf6-eb07-4d03-b82f-b6a28f99c43f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:04.921721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f316baf6-eb07-4d03-b82f-b6a28f99c43f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:04.923393 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1022/2079] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:04 2022] DELETE /identity/v3/projects/bf1b4c4e32ba4f8a8fdda48b930d55fa => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:04.928492 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f19bef-cadf-436d-aeff-ba2af71bcf62 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:04.928846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f19bef-cadf-436d-aeff-ba2af71bcf62 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:04.929140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f19bef-cadf-436d-aeff-ba2af71bcf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:04.950030 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b7f19bef-cadf-436d-aeff-ba2af71bcf62 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:04.951284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b7f19bef-cadf-436d-aeff-ba2af71bcf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:04.992176 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:05.027459 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1058/2080] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:38:05.036180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7823c-b903-4b25-8e41-36eb259a12d9 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:05.036611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7823c-b903-4b25-8e41-36eb259a12d9 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:05.037293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7823c-b903-4b25-8e41-36eb259a12d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:05.059708 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-82d7823c-b903-4b25-8e41-36eb259a12d9 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:05.060985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-82d7823c-b903-4b25-8e41-36eb259a12d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:05.071095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1023/2081] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:05.075354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f19bef-cadf-436d-aeff-ba2af71bcf62 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:05.092588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f19bef-cadf-436d-aeff-ba2af71bcf62 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:05.092588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b171f44-9681-4a3e-9887-23ee51cdd41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:05.092939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b171f44-9681-4a3e-9887-23ee51cdd41f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:05.093141 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b171f44-9681-4a3e-9887-23ee51cdd41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:05.093403 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b171f44-9681-4a3e-9887-23ee51cdd41f admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:05.094051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b171f44-9681-4a3e-9887-23ee51cdd41f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:38:05.094351 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b171f44-9681-4a3e-9887-23ee51cdd41f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:05.095992 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b171f44-9681-4a3e-9887-23ee51cdd41f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:05.110168 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1059/2082] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:38:05.114622 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7823c-b903-4b25-8e41-36eb259a12d9 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:05.124899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7823c-b903-4b25-8e41-36eb259a12d9 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:05.129625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82156170-3797-4859-9859-7ffcd446a329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:05.130255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82156170-3797-4859-9859-7ffcd446a329 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:05.130532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82156170-3797-4859-9859-7ffcd446a329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:05.130916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82156170-3797-4859-9859-7ffcd446a329 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:05.131441 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82156170-3797-4859-9859-7ffcd446a329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:05.134440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82156170-3797-4859-9859-7ffcd446a329 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:05.166303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1024/2083] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:05 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:05.170742 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b171f44-9681-4a3e-9887-23ee51cdd41f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:05.181383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b171f44-9681-4a3e-9887-23ee51cdd41f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:05.241188 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c44cf5-d3b0-433f-b1d6-83aab2e8615c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:05.241900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c44cf5-d3b0-433f-b1d6-83aab2e8615c admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:05.242277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c44cf5-d3b0-433f-b1d6-83aab2e8615c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:05.242726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c44cf5-d3b0-433f-b1d6-83aab2e8615c admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:05.243531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c44cf5-d3b0-433f-b1d6-83aab2e8615c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:05.245405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c44cf5-d3b0-433f-b1d6-83aab2e8615c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:05.274545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1060/2084] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:05 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:05.277008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82156170-3797-4859-9859-7ffcd446a329 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:05.287451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82156170-3797-4859-9859-7ffcd446a329 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:05.294837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ea3ed-ac35-4e46-8892-30bd2c565709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:05.295539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ea3ed-ac35-4e46-8892-30bd2c565709 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:05.295914 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ea3ed-ac35-4e46-8892-30bd2c565709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:05.296281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ea3ed-ac35-4e46-8892-30bd2c565709 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:05.297103 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ea3ed-ac35-4e46-8892-30bd2c565709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:05.298802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ea3ed-ac35-4e46-8892-30bd2c565709 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:05.309765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1025/2085] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:38:05.313914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c44cf5-d3b0-433f-b1d6-83aab2e8615c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:05.324713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c44cf5-d3b0-433f-b1d6-83aab2e8615c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:05.330449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025157c-7806-4b2a-a19c-d462f17d7240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:05.331187 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025157c-7806-4b2a-a19c-d462f17d7240 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:05.331523 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025157c-7806-4b2a-a19c-d462f17d7240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:05.331856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025157c-7806-4b2a-a19c-d462f17d7240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:05.332617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025157c-7806-4b2a-a19c-d462f17d7240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:05.334390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025157c-7806-4b2a-a19c-d462f17d7240 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:05.346300 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1061/2086] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:05.350834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ea3ed-ac35-4e46-8892-30bd2c565709 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:05.363631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ea3ed-ac35-4e46-8892-30bd2c565709 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:05.368970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2dc0af-af77-4c01-8593-ed7ca8412588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:05.369638 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2dc0af-af77-4c01-8593-ed7ca8412588 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:05.369910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2dc0af-af77-4c01-8593-ed7ca8412588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:05.370177 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2dc0af-af77-4c01-8593-ed7ca8412588 admin admin] PATH_INFO: `/v3/projects/b3c33dc11d9d41eabd986b4a4c536616/users/535e932c54904b83bfc19fd68df95116/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:05.372783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:05.388958 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2dc0af-af77-4c01-8593-ed7ca8412588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c33dc11d9d41eabd986b4a4c536616, user_id=535e932c54904b83bfc19fd68df95116, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:05.392257 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2dc0af-af77-4c01-8593-ed7ca8412588 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:05.410459 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1026/2087] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:05 2022] PUT /identity/v3/projects/b3c33dc11d9d41eabd986b4a4c536616/users/535e932c54904b83bfc19fd68df95116/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:05.421407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f017c8e-5b86-4317-85da-c953cbc2151d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:05.421717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f017c8e-5b86-4317-85da-c953cbc2151d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:05.422019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f017c8e-5b86-4317-85da-c953cbc2151d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:05.453254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7f017c8e-5b86-4317-85da-c953cbc2151d None None] MFA Rules not processed for user `535e932c54904b83bfc19fd68df95116`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:05.454528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7f017c8e-5b86-4317-85da-c953cbc2151d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:05.528591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1062/2088] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:05.543345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2dc0af-af77-4c01-8593-ed7ca8412588 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:05.545486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-bf2dc0af-af77-4c01-8593-ed7ca8412588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:05.613661 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2dc0af-af77-4c01-8593-ed7ca8412588 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:05.621500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701e15e-a625-499a-b0a9-1b4346e10ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:05.622292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701e15e-a625-499a-b0a9-1b4346e10ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:05.622694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701e15e-a625-499a-b0a9-1b4346e10ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:05.623036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701e15e-a625-499a-b0a9-1b4346e10ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:05.633594 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701e15e-a625-499a-b0a9-1b4346e10ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:05.635055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701e15e-a625-499a-b0a9-1b4346e10ea3 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:05.644859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1027/2089] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:09.190063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9ba24-6944-4e5c-94aa-084216240f1e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:09.190621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9ba24-6944-4e5c-94aa-084216240f1e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:09.190621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9ba24-6944-4e5c-94aa-084216240f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:09.211651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-aee9ba24-6944-4e5c-94aa-084216240f1e None None] MFA Rules not processed for user `535e932c54904b83bfc19fd68df95116`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:09.212939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-aee9ba24-6944-4e5c-94aa-084216240f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:09.221604 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1063/2090] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:15.640167 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:19.136228 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701e15e-a625-499a-b0a9-1b4346e10ea3 service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.138386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9701e15e-a625-499a-b0a9-1b4346e10ea3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:19.206698 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:19.231646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701e15e-a625-499a-b0a9-1b4346e10ea3 service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.238803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d06b2-85f5-4768-a047-f6231583f6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.239613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d06b2-85f5-4768-a047-f6231583f6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.239931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d06b2-85f5-4768-a047-f6231583f6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.240197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d06b2-85f5-4768-a047-f6231583f6f7 admin admin] PATH_INFO: `/v3/users/64357433676a45a9b6cc2ee2b19f33a1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.241614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d06b2-85f5-4768-a047-f6231583f6f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64357433676a45a9b6cc2ee2b19f33a1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.243757 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d06b2-85f5-4768-a047-f6231583f6f7 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:19.317741 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1c7d06b2-85f5-4768-a047-f6231583f6f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64357433676a45a9b6cc2ee2b19f33a1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:19.347162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1c7d06b2-85f5-4768-a047-f6231583f6f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64357433676a45a9b6cc2ee2b19f33a1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:19.359933 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1c7d06b2-85f5-4768-a047-f6231583f6f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64357433676a45a9b6cc2ee2b19f33a1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:19.375870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1c7d06b2-85f5-4768-a047-f6231583f6f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64357433676a45a9b6cc2ee2b19f33a1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:19.377926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1c7d06b2-85f5-4768-a047-f6231583f6f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64357433676a45a9b6cc2ee2b19f33a1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:19.387697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1028/2091] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:19 2022] DELETE /identity/v3/users/64357433676a45a9b6cc2ee2b19f33a1 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:19.393618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adcba4-befa-4e7a-835c-74be185f4402 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.393901 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adcba4-befa-4e7a-835c-74be185f4402 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.394173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adcba4-befa-4e7a-835c-74be185f4402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.423453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-03adcba4-befa-4e7a-835c-74be185f4402 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:19.425335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-03adcba4-befa-4e7a-835c-74be185f4402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:19.515661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1064/2092] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:19.527381 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d06b2-85f5-4768-a047-f6231583f6f7 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.529555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1c7d06b2-85f5-4768-a047-f6231583f6f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:19.596185 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d06b2-85f5-4768-a047-f6231583f6f7 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.600239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226cf9e-6075-4177-9ee4-bc892738610d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.601315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226cf9e-6075-4177-9ee4-bc892738610d service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.601315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226cf9e-6075-4177-9ee4-bc892738610d service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.601521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226cf9e-6075-4177-9ee4-bc892738610d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.614496 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226cf9e-6075-4177-9ee4-bc892738610d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.616022 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226cf9e-6075-4177-9ee4-bc892738610d service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:19.626780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1029/2093] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:19.944789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03adcba4-befa-4e7a-835c-74be185f4402 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:19.946299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-03adcba4-befa-4e7a-835c-74be185f4402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:19.965379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03adcba4-befa-4e7a-835c-74be185f4402 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:19.975175 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa84a73-4667-49ed-a345-72764e2121f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:19.976165 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa84a73-4667-49ed-a345-72764e2121f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.976480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa84a73-4667-49ed-a345-72764e2121f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.976788 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa84a73-4667-49ed-a345-72764e2121f0 admin admin] PATH_INFO: `/v3/projects/45f669183769418a91b495a85c37d4c3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.977921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa84a73-4667-49ed-a345-72764e2121f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45f669183769418a91b495a85c37d4c3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:19.980913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa84a73-4667-49ed-a345-72764e2121f0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:20.042430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eaa84a73-4667-49ed-a345-72764e2121f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45f669183769418a91b495a85c37d4c3', 'request_id': 'req-eaa84a73-4667-49ed-a345-72764e2121f0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:20.052837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eaa84a73-4667-49ed-a345-72764e2121f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45f669183769418a91b495a85c37d4c3', 'request_id': 'req-eaa84a73-4667-49ed-a345-72764e2121f0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:20.064349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eaa84a73-4667-49ed-a345-72764e2121f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45f669183769418a91b495a85c37d4c3', 'request_id': 'req-eaa84a73-4667-49ed-a345-72764e2121f0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:20.083088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eaa84a73-4667-49ed-a345-72764e2121f0 admin admin] The token cache is being invalidate because project 45f669183769418a91b495a85c37d4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:20.083384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eaa84a73-4667-49ed-a345-72764e2121f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:20.084221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eaa84a73-4667-49ed-a345-72764e2121f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:20.086764 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1065/2094] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:19 2022] DELETE /identity/v3/projects/45f669183769418a91b495a85c37d4c3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:20.092271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733280a6-e958-432a-a741-ef1ccc807e0b None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:20.092638 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733280a6-e958-432a-a741-ef1ccc807e0b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:20.093042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733280a6-e958-432a-a741-ef1ccc807e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:20.117955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-733280a6-e958-432a-a741-ef1ccc807e0b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:20.119165 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-733280a6-e958-432a-a741-ef1ccc807e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:20.138227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:20.201712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1030/2095] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:20.211075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2671c70-3b74-4591-8c02-14d673901c33 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:20.211547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2671c70-3b74-4591-8c02-14d673901c33 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:20.211626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2671c70-3b74-4591-8c02-14d673901c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:20.235276 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a2671c70-3b74-4591-8c02-14d673901c33 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:20.236579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a2671c70-3b74-4591-8c02-14d673901c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:20.247032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1066/2096] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:20.251287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733280a6-e958-432a-a741-ef1ccc807e0b None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:20.264701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733280a6-e958-432a-a741-ef1ccc807e0b None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:20.272839 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497ddeb-826b-4866-b946-f2860f5ceb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:20.274182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497ddeb-826b-4866-b946-f2860f5ceb39 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:20.274311 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497ddeb-826b-4866-b946-f2860f5ceb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:20.274585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497ddeb-826b-4866-b946-f2860f5ceb39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:20.275244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497ddeb-826b-4866-b946-f2860f5ceb39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:38:20.275596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497ddeb-826b-4866-b946-f2860f5ceb39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:20.282060 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497ddeb-826b-4866-b946-f2860f5ceb39 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:20.302629 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1031/2097] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:38:20.308314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2671c70-3b74-4591-8c02-14d673901c33 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:20.317693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2671c70-3b74-4591-8c02-14d673901c33 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:20.322286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fd912-ae7c-42c6-a779-0819b13889f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:20.322998 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fd912-ae7c-42c6-a779-0819b13889f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:20.323295 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fd912-ae7c-42c6-a779-0819b13889f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:20.323530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fd912-ae7c-42c6-a779-0819b13889f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:20.324197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fd912-ae7c-42c6-a779-0819b13889f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:20.326089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fd912-ae7c-42c6-a779-0819b13889f8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:20.366326 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1067/2098] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:20 2022] POST /identity/v3/projects => generated 391 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:20.371945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497ddeb-826b-4866-b946-f2860f5ceb39 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:20.387289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497ddeb-826b-4866-b946-f2860f5ceb39 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:20.529479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff1f1a-a373-4e51-bb34-0f63e6670557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:20.530321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff1f1a-a373-4e51-bb34-0f63e6670557 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:20.530652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff1f1a-a373-4e51-bb34-0f63e6670557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:20.530918 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff1f1a-a373-4e51-bb34-0f63e6670557 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:20.536280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff1f1a-a373-4e51-bb34-0f63e6670557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:20.538331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff1f1a-a373-4e51-bb34-0f63e6670557 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:20.618792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1032/2099] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:20 2022] POST /identity/v3/users => generated 296 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:20.624655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fd912-ae7c-42c6-a779-0819b13889f8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:20.643161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fd912-ae7c-42c6-a779-0819b13889f8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:20.649667 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13294e38-1eab-4cc4-a832-60546c229e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:20.650475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13294e38-1eab-4cc4-a832-60546c229e19 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:20.650746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13294e38-1eab-4cc4-a832-60546c229e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:20.651013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13294e38-1eab-4cc4-a832-60546c229e19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:20.651815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13294e38-1eab-4cc4-a832-60546c229e19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:20.653635 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13294e38-1eab-4cc4-a832-60546c229e19 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:20.667304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1068/2100] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:38:20.670145 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff1f1a-a373-4e51-bb34-0f63e6670557 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:20.688138 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff1f1a-a373-4e51-bb34-0f63e6670557 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:20.693709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a087577-4676-41d2-9393-d9faefd5004a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:20.694409 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a087577-4676-41d2-9393-d9faefd5004a admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:20.694741 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a087577-4676-41d2-9393-d9faefd5004a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:20.695064 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a087577-4676-41d2-9393-d9faefd5004a admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:20.695707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a087577-4676-41d2-9393-d9faefd5004a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:20.697396 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a087577-4676-41d2-9393-d9faefd5004a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:20.709598 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1033/2101] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:38:20.714872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13294e38-1eab-4cc4-a832-60546c229e19 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:20.729424 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13294e38-1eab-4cc4-a832-60546c229e19 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:20.734709 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656300fa-8774-4c7b-b2e0-20d87c8b7a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:20.735486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656300fa-8774-4c7b-b2e0-20d87c8b7a1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:20.735818 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656300fa-8774-4c7b-b2e0-20d87c8b7a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:20.736290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656300fa-8774-4c7b-b2e0-20d87c8b7a1a admin admin] PATH_INFO: `/v3/projects/843586eece33415f8dcc8d5060105ef7/users/4c34c00b35da4fbc9f9d045c0f07d58b/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:20.752549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656300fa-8774-4c7b-b2e0-20d87c8b7a1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=843586eece33415f8dcc8d5060105ef7, user_id=4c34c00b35da4fbc9f9d045c0f07d58b, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:20.755440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656300fa-8774-4c7b-b2e0-20d87c8b7a1a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:20.776392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1069/2102] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:20 2022] PUT /identity/v3/projects/843586eece33415f8dcc8d5060105ef7/users/4c34c00b35da4fbc9f9d045c0f07d58b/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:20.787029 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d50cf-54e3-49dc-be6c-603792922d43 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:20.787199 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d50cf-54e3-49dc-be6c-603792922d43 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:20.787466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d50cf-54e3-49dc-be6c-603792922d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:20.829348 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-3b2d50cf-54e3-49dc-be6c-603792922d43 None None] MFA Rules not processed for user `4c34c00b35da4fbc9f9d045c0f07d58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:20.830947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3b2d50cf-54e3-49dc-be6c-603792922d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:20.958472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1034/2103] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:20.972340 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656300fa-8774-4c7b-b2e0-20d87c8b7a1a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:20.977272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-656300fa-8774-4c7b-b2e0-20d87c8b7a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:21.016774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:21.075988 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656300fa-8774-4c7b-b2e0-20d87c8b7a1a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:21.081293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc0653-f697-45a9-8be9-809756787872 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:21.082056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc0653-f697-45a9-8be9-809756787872 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:21.082287 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc0653-f697-45a9-8be9-809756787872 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:21.082523 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc0653-f697-45a9-8be9-809756787872 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:21.105197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc0653-f697-45a9-8be9-809756787872 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:21.107011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc0653-f697-45a9-8be9-809756787872 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:21.118639 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1070/2104] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:30.945507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:31.112701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=17 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:37.490679 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d50cf-54e3-49dc-be6c-603792922d43 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:37.493265 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3b2d50cf-54e3-49dc-be6c-603792922d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:37.573249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d50cf-54e3-49dc-be6c-603792922d43 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:37.578905 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0399d0-0328-4acd-935a-d29622a2d402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:37.579567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0399d0-0328-4acd-935a-d29622a2d402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:37.579813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0399d0-0328-4acd-935a-d29622a2d402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:37.580141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0399d0-0328-4acd-935a-d29622a2d402 admin admin] PATH_INFO: `/v3/users/bdb3c71e5ef3418cae4045b6dab1340f` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:37.581358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0399d0-0328-4acd-935a-d29622a2d402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdb3c71e5ef3418cae4045b6dab1340f)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:37.583238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0399d0-0328-4acd-935a-d29622a2d402 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:37.645301 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5e0399d0-0328-4acd-935a-d29622a2d402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdb3c71e5ef3418cae4045b6dab1340f'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:37.661536 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5e0399d0-0328-4acd-935a-d29622a2d402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdb3c71e5ef3418cae4045b6dab1340f'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:37.678538 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5e0399d0-0328-4acd-935a-d29622a2d402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdb3c71e5ef3418cae4045b6dab1340f'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:37.689074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5e0399d0-0328-4acd-935a-d29622a2d402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdb3c71e5ef3418cae4045b6dab1340f'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:37.690098 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5e0399d0-0328-4acd-935a-d29622a2d402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdb3c71e5ef3418cae4045b6dab1340f'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:37.698668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1035/2105] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:37 2022] DELETE /identity/v3/users/bdb3c71e5ef3418cae4045b6dab1340f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:37.705431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd67d0e-3926-4cf3-be96-87ad7ea97a24 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:37.705890 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd67d0e-3926-4cf3-be96-87ad7ea97a24 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:37.706343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd67d0e-3926-4cf3-be96-87ad7ea97a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:37.733167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dfd67d0e-3926-4cf3-be96-87ad7ea97a24 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:37.734601 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dfd67d0e-3926-4cf3-be96-87ad7ea97a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:37.806975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1071/2106] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:37.817957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0399d0-0328-4acd-935a-d29622a2d402 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:37.819325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5e0399d0-0328-4acd-935a-d29622a2d402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:37.881102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0399d0-0328-4acd-935a-d29622a2d402 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:37.885148 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0525b-d78d-4d9f-8814-45149519b15b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:37.885775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0525b-d78d-4d9f-8814-45149519b15b service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:37.886030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0525b-d78d-4d9f-8814-45149519b15b service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:37.886360 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0525b-d78d-4d9f-8814-45149519b15b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:37.896525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0525b-d78d-4d9f-8814-45149519b15b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:37.897968 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0525b-d78d-4d9f-8814-45149519b15b service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:37.908836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1036/2107] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:38.649036 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd67d0e-3926-4cf3-be96-87ad7ea97a24 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:38.654930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dfd67d0e-3926-4cf3-be96-87ad7ea97a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:38.688884 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd67d0e-3926-4cf3-be96-87ad7ea97a24 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:38.694368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:38.695185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:38.695621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:38.695924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] PATH_INFO: `/v3/projects/237b1c318adb4cf4a5f4b3685af554e7` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:38.697360 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=237b1c318adb4cf4a5f4b3685af554e7)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:38.700936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:38.787027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '237b1c318adb4cf4a5f4b3685af554e7', 'request_id': 'req-0bcb9738-26f3-43ae-a006-20e1815340a3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:38.799562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '237b1c318adb4cf4a5f4b3685af554e7', 'request_id': 'req-0bcb9738-26f3-43ae-a006-20e1815340a3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:38.810644 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '237b1c318adb4cf4a5f4b3685af554e7', 'request_id': 'req-0bcb9738-26f3-43ae-a006-20e1815340a3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:38.829167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] The token cache is being invalidate because project 237b1c318adb4cf4a5f4b3685af554e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:38.829454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:38.830168 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:38.831926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1072/2108] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:38 2022] DELETE /identity/v3/projects/237b1c318adb4cf4a5f4b3685af554e7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:38.835920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0525b-d78d-4d9f-8814-45149519b15b service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:38.848411 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0525b-d78d-4d9f-8814-45149519b15b service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:38.892529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:38.947535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ff580-2443-4b1e-b16c-9624f6040d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:38.948240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ff580-2443-4b1e-b16c-9624f6040d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:38.948514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ff580-2443-4b1e-b16c-9624f6040d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:38.948784 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ff580-2443-4b1e-b16c-9624f6040d85 admin admin] PATH_INFO: `/v3/users/edde78efe3b1483b9c90843801662ef1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:38.950026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ff580-2443-4b1e-b16c-9624f6040d85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edde78efe3b1483b9c90843801662ef1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:38.953141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ff580-2443-4b1e-b16c-9624f6040d85 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:39.047408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e17ff580-2443-4b1e-b16c-9624f6040d85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edde78efe3b1483b9c90843801662ef1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:39.075865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e17ff580-2443-4b1e-b16c-9624f6040d85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edde78efe3b1483b9c90843801662ef1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:39.093313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e17ff580-2443-4b1e-b16c-9624f6040d85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edde78efe3b1483b9c90843801662ef1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:39.112588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e17ff580-2443-4b1e-b16c-9624f6040d85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edde78efe3b1483b9c90843801662ef1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:39.112771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e17ff580-2443-4b1e-b16c-9624f6040d85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edde78efe3b1483b9c90843801662ef1'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:39.124416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1037/2109] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:38 2022] DELETE /identity/v3/users/edde78efe3b1483b9c90843801662ef1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:39.532940 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:39.534670 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:39.549019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb9738-26f3-43ae-a006-20e1815340a3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:39.609659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:39.611479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:39.611479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:39.611479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0 admin admin] PATH_INFO: `/v3/projects/930307745e1948d1a0da371aa4a1276d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:39.612080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=930307745e1948d1a0da371aa4a1276d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:39.614009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:39.649229 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:39.676734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '930307745e1948d1a0da371aa4a1276d', 'request_id': 'req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:39.687079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '930307745e1948d1a0da371aa4a1276d', 'request_id': 'req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:39.709109 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '930307745e1948d1a0da371aa4a1276d', 'request_id': 'req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:39.726309 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0 admin admin] The token cache is being invalidate because project 930307745e1948d1a0da371aa4a1276d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:39.727112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:39.728198 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-34c0c05e-621d-4b8a-b7aa-20710f1e94e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:39.730445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1073/2110] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:39 2022] DELETE /identity/v3/projects/930307745e1948d1a0da371aa4a1276d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:39.736175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab538b0f-5e33-425c-aaba-cbdf6a92520e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:39.736476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab538b0f-5e33-425c-aaba-cbdf6a92520e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:39.736752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab538b0f-5e33-425c-aaba-cbdf6a92520e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:39.760669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-ab538b0f-5e33-425c-aaba-cbdf6a92520e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:39.766637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ab538b0f-5e33-425c-aaba-cbdf6a92520e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:39.853325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1038/2111] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:39.865108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b55fb2-5d2a-4528-a48c-6297b4755f8e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:39.866003 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b55fb2-5d2a-4528-a48c-6297b4755f8e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:39.866504 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b55fb2-5d2a-4528-a48c-6297b4755f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:39.907587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-16b55fb2-5d2a-4528-a48c-6297b4755f8e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:39.908753 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-16b55fb2-5d2a-4528-a48c-6297b4755f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:39.921434 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1074/2112] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:39.923056 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab538b0f-5e33-425c-aaba-cbdf6a92520e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:39.933150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab538b0f-5e33-425c-aaba-cbdf6a92520e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:39.938004 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23890af7-9c5f-4897-ac81-dee44f7cb45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:39.938645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23890af7-9c5f-4897-ac81-dee44f7cb45f admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:39.938923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23890af7-9c5f-4897-ac81-dee44f7cb45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:39.939188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23890af7-9c5f-4897-ac81-dee44f7cb45f admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:39.940002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23890af7-9c5f-4897-ac81-dee44f7cb45f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:38:39.940301 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23890af7-9c5f-4897-ac81-dee44f7cb45f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:39.942081 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23890af7-9c5f-4897-ac81-dee44f7cb45f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:39.956103 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1039/2113] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:38:39.960688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b55fb2-5d2a-4528-a48c-6297b4755f8e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:39.970621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b55fb2-5d2a-4528-a48c-6297b4755f8e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:39.975506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de71fa0-a3b5-4c46-8080-36720991ab7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:39.976167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de71fa0-a3b5-4c46-8080-36720991ab7b admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:39.976444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de71fa0-a3b5-4c46-8080-36720991ab7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:39.976712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de71fa0-a3b5-4c46-8080-36720991ab7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:39.977351 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de71fa0-a3b5-4c46-8080-36720991ab7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:39.979247 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de71fa0-a3b5-4c46-8080-36720991ab7b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:40.024247 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1075/2114] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:39 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:38:40.028173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23890af7-9c5f-4897-ac81-dee44f7cb45f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:40.041711 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23890af7-9c5f-4897-ac81-dee44f7cb45f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:40.073044 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:40.117973 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b11ea-2372-43e9-80d1-4ba0548f2d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:40.118847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b11ea-2372-43e9-80d1-4ba0548f2d1a admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:40.119259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b11ea-2372-43e9-80d1-4ba0548f2d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:40.119666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b11ea-2372-43e9-80d1-4ba0548f2d1a admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:40.120660 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b11ea-2372-43e9-80d1-4ba0548f2d1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:40.124036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b11ea-2372-43e9-80d1-4ba0548f2d1a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:40.153847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1040/2115] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:40 2022] POST /identity/v3/users => generated 286 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:40.160441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de71fa0-a3b5-4c46-8080-36720991ab7b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:40.175801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de71fa0-a3b5-4c46-8080-36720991ab7b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:40.182815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19cdbe-05d5-48c6-8c95-35990c12de7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:40.183705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19cdbe-05d5-48c6-8c95-35990c12de7d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:40.184156 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19cdbe-05d5-48c6-8c95-35990c12de7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:40.184579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19cdbe-05d5-48c6-8c95-35990c12de7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:40.185754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19cdbe-05d5-48c6-8c95-35990c12de7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:40.188328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19cdbe-05d5-48c6-8c95-35990c12de7d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:40.202623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1076/2116] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:40.207010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b11ea-2372-43e9-80d1-4ba0548f2d1a admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:40.221366 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b11ea-2372-43e9-80d1-4ba0548f2d1a admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:40.228327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b63cc-575a-4af8-b793-a722a8c749e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:40.228327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b63cc-575a-4af8-b793-a722a8c749e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:40.228327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b63cc-575a-4af8-b793-a722a8c749e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:40.228863 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b63cc-575a-4af8-b793-a722a8c749e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:40.229597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b63cc-575a-4af8-b793-a722a8c749e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:40.231272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b63cc-575a-4af8-b793-a722a8c749e4 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:40.253854 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1041/2117] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:40.258666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19cdbe-05d5-48c6-8c95-35990c12de7d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:40.275085 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19cdbe-05d5-48c6-8c95-35990c12de7d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:40.282160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0013f-54cf-403f-b529-cb8f87a565a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:40.282845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0013f-54cf-403f-b529-cb8f87a565a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:40.283119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0013f-54cf-403f-b529-cb8f87a565a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:40.283385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0013f-54cf-403f-b529-cb8f87a565a0 admin admin] PATH_INFO: `/v3/projects/cfc7d3f0e5104028a60f5a70900b5eb5/users/7c2516fd8fdd429982be5e6495d3ad3e/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:40.300851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0013f-54cf-403f-b529-cb8f87a565a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc7d3f0e5104028a60f5a70900b5eb5, user_id=7c2516fd8fdd429982be5e6495d3ad3e, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:40.303876 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0013f-54cf-403f-b529-cb8f87a565a0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:40.322706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1077/2118] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:40 2022] PUT /identity/v3/projects/cfc7d3f0e5104028a60f5a70900b5eb5/users/7c2516fd8fdd429982be5e6495d3ad3e/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:40.346375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e11948-ed63-4cf3-8c3d-21ed3680758d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:40.346972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e11948-ed63-4cf3-8c3d-21ed3680758d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:40.347470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e11948-ed63-4cf3-8c3d-21ed3680758d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:40.390635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-89e11948-ed63-4cf3-8c3d-21ed3680758d None None] MFA Rules not processed for user `7c2516fd8fdd429982be5e6495d3ad3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:40.391837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-89e11948-ed63-4cf3-8c3d-21ed3680758d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:40.459493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1042/2119] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:40.465325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0013f-54cf-403f-b529-cb8f87a565a0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:40.541466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0013f-54cf-403f-b529-cb8f87a565a0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:40.546924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaebe07-afba-40ae-8508-5ac04734702d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:40.547603 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaebe07-afba-40ae-8508-5ac04734702d admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:40.547944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaebe07-afba-40ae-8508-5ac04734702d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:40.548221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaebe07-afba-40ae-8508-5ac04734702d admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:40.548862 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaebe07-afba-40ae-8508-5ac04734702d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:40.551295 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaebe07-afba-40ae-8508-5ac04734702d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:40.591935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1078/2120] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:40 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:40.595132 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e11948-ed63-4cf3-8c3d-21ed3680758d None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:40.608585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e11948-ed63-4cf3-8c3d-21ed3680758d None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:40.688649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0885260-7b52-4666-baa6-68931729a9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:40.689352 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0885260-7b52-4666-baa6-68931729a9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:40.689713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0885260-7b52-4666-baa6-68931729a9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:40.690047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0885260-7b52-4666-baa6-68931729a9c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:40.690761 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0885260-7b52-4666-baa6-68931729a9c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:40.693238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0885260-7b52-4666-baa6-68931729a9c2 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:40.719849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1043/2121] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:40 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:40.724073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaebe07-afba-40ae-8508-5ac04734702d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:40.735510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaebe07-afba-40ae-8508-5ac04734702d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:40.740217 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a6815-5009-404b-a013-9577f8430a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:40.740826 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a6815-5009-404b-a013-9577f8430a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:40.741104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a6815-5009-404b-a013-9577f8430a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:40.741372 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a6815-5009-404b-a013-9577f8430a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:40.742134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a6815-5009-404b-a013-9577f8430a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:40.743772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a6815-5009-404b-a013-9577f8430a99 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:40.759417 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1079/2122] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:38:40.760586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0885260-7b52-4666-baa6-68931729a9c2 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:40.771014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0885260-7b52-4666-baa6-68931729a9c2 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:40.775606 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79d059-d132-466b-b79f-94d514311c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:40.776296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79d059-d132-466b-b79f-94d514311c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:40.776567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79d059-d132-466b-b79f-94d514311c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:40.776830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79d059-d132-466b-b79f-94d514311c4b admin admin] PATH_INFO: `/v3/projects/1dc197b087c843e38552f19cdf95b651/users/8f166fa29e7d4864af80c8f21d9c3454/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:40.794840 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79d059-d132-466b-b79f-94d514311c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc197b087c843e38552f19cdf95b651, user_id=8f166fa29e7d4864af80c8f21d9c3454, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:40.797576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79d059-d132-466b-b79f-94d514311c4b admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:40.812880 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1044/2123] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:40 2022] PUT /identity/v3/projects/1dc197b087c843e38552f19cdf95b651/users/8f166fa29e7d4864af80c8f21d9c3454/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:38:40.817003 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a6815-5009-404b-a013-9577f8430a99 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:40.828018 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a6815-5009-404b-a013-9577f8430a99 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:40.849190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:40.889514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22ea52-d3c1-42a3-8e9d-159edb51705f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:40.890182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22ea52-d3c1-42a3-8e9d-159edb51705f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:40.890382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22ea52-d3c1-42a3-8e9d-159edb51705f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:40.890645 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22ea52-d3c1-42a3-8e9d-159edb51705f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:40.893580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22ea52-d3c1-42a3-8e9d-159edb51705f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:40.893580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22ea52-d3c1-42a3-8e9d-159edb51705f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:40.903047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1080/2124] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:40.907395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79d059-d132-466b-b79f-94d514311c4b admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:40.913042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:40.918800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79d059-d132-466b-b79f-94d514311c4b admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:40.924703 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ac19b-2b0e-4c9e-9f7e-132b8a4ac389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:40.925816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ac19b-2b0e-4c9e-9f7e-132b8a4ac389 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:40.926472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ac19b-2b0e-4c9e-9f7e-132b8a4ac389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:40.926582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ac19b-2b0e-4c9e-9f7e-132b8a4ac389 admin admin] PATH_INFO: `/v3/projects/1dc197b087c843e38552f19cdf95b651/users/8f166fa29e7d4864af80c8f21d9c3454/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:40.928621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ac19b-2b0e-4c9e-9f7e-132b8a4ac389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc197b087c843e38552f19cdf95b651, user_id=8f166fa29e7d4864af80c8f21d9c3454, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:40.931379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ac19b-2b0e-4c9e-9f7e-132b8a4ac389 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:40.944511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1045/2125] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:40 2022] PUT /identity/v3/projects/1dc197b087c843e38552f19cdf95b651/users/8f166fa29e7d4864af80c8f21d9c3454/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:38:40.956574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28ad1e-f73e-46fb-8a3a-0783e36a79bb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:40.956574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28ad1e-f73e-46fb-8a3a-0783e36a79bb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:40.956842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28ad1e-f73e-46fb-8a3a-0783e36a79bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:40.989112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bd28ad1e-f73e-46fb-8a3a-0783e36a79bb None None] MFA Rules not processed for user `8f166fa29e7d4864af80c8f21d9c3454`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:40.990550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bd28ad1e-f73e-46fb-8a3a-0783e36a79bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:41.070671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1081/2126] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:41.083307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ac19b-2b0e-4c9e-9f7e-132b8a4ac389 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:41.085096 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-be0ac19b-2b0e-4c9e-9f7e-132b8a4ac389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:41.155462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ac19b-2b0e-4c9e-9f7e-132b8a4ac389 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:41.160176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a9503-856c-4943-992e-17e7f35e3ec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:41.160795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a9503-856c-4943-992e-17e7f35e3ec2 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:41.161075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a9503-856c-4943-992e-17e7f35e3ec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:41.161347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a9503-856c-4943-992e-17e7f35e3ec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:41.170846 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a9503-856c-4943-992e-17e7f35e3ec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:41.172328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a9503-856c-4943-992e-17e7f35e3ec2 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:41.182180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1046/2127] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:41.394469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28ad1e-f73e-46fb-8a3a-0783e36a79bb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:41.396053 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bd28ad1e-f73e-46fb-8a3a-0783e36a79bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:41.482318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28ad1e-f73e-46fb-8a3a-0783e36a79bb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:41.488905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394e525-7f27-42b2-af6b-7eb7e04511dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:41.490966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394e525-7f27-42b2-af6b-7eb7e04511dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:41.491353 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394e525-7f27-42b2-af6b-7eb7e04511dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:41.491766 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394e525-7f27-42b2-af6b-7eb7e04511dd admin admin] PATH_INFO: `/v3/users/4c34c00b35da4fbc9f9d045c0f07d58b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:41.493173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394e525-7f27-42b2-af6b-7eb7e04511dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c34c00b35da4fbc9f9d045c0f07d58b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:41.495695 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394e525-7f27-42b2-af6b-7eb7e04511dd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:41.561881 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9394e525-7f27-42b2-af6b-7eb7e04511dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c34c00b35da4fbc9f9d045c0f07d58b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:41.564683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:41.573080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9394e525-7f27-42b2-af6b-7eb7e04511dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c34c00b35da4fbc9f9d045c0f07d58b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:41.574169 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9394e525-7f27-42b2-af6b-7eb7e04511dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c34c00b35da4fbc9f9d045c0f07d58b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:41.587707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9394e525-7f27-42b2-af6b-7eb7e04511dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c34c00b35da4fbc9f9d045c0f07d58b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:41.595009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9394e525-7f27-42b2-af6b-7eb7e04511dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c34c00b35da4fbc9f9d045c0f07d58b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:41.601208 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1082/2128] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:41 2022] DELETE /identity/v3/users/4c34c00b35da4fbc9f9d045c0f07d58b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:41.605813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49d7f8-16a8-4dda-acfe-8ba8d700113d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:41.606088 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49d7f8-16a8-4dda-acfe-8ba8d700113d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:41.608315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49d7f8-16a8-4dda-acfe-8ba8d700113d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:41.627536 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-9f49d7f8-16a8-4dda-acfe-8ba8d700113d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:41.628797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9f49d7f8-16a8-4dda-acfe-8ba8d700113d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:41.649053 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394e525-7f27-42b2-af6b-7eb7e04511dd admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:41.650909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9394e525-7f27-42b2-af6b-7eb7e04511dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:41.705180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1047/2129] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:41.715251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49d7f8-16a8-4dda-acfe-8ba8d700113d None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:41.717044 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9f49d7f8-16a8-4dda-acfe-8ba8d700113d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:41.752295 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394e525-7f27-42b2-af6b-7eb7e04511dd admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:41.775391 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d203da0-e8e4-4849-b6db-5981aba44f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:41.776333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d203da0-e8e4-4849-b6db-5981aba44f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:41.776679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d203da0-e8e4-4849-b6db-5981aba44f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:41.777020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d203da0-e8e4-4849-b6db-5981aba44f22 admin admin] PATH_INFO: `/v3/users/7c2516fd8fdd429982be5e6495d3ad3e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:41.779996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d203da0-e8e4-4849-b6db-5981aba44f22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2516fd8fdd429982be5e6495d3ad3e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:41.782300 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d203da0-e8e4-4849-b6db-5981aba44f22 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:41.797347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49d7f8-16a8-4dda-acfe-8ba8d700113d None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:41.801813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db30db-d3d4-4291-952d-84336e52d243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:41.802663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db30db-d3d4-4291-952d-84336e52d243 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:41.803175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db30db-d3d4-4291-952d-84336e52d243 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:41.803518 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db30db-d3d4-4291-952d-84336e52d243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:41.817499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db30db-d3d4-4291-952d-84336e52d243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:41.819151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db30db-d3d4-4291-952d-84336e52d243 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:41.839780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1d203da0-e8e4-4849-b6db-5981aba44f22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2516fd8fdd429982be5e6495d3ad3e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:41.842261 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1048/2130] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:38:41.850626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1d203da0-e8e4-4849-b6db-5981aba44f22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2516fd8fdd429982be5e6495d3ad3e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:41.851614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1d203da0-e8e4-4849-b6db-5981aba44f22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2516fd8fdd429982be5e6495d3ad3e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:41.868889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1d203da0-e8e4-4849-b6db-5981aba44f22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2516fd8fdd429982be5e6495d3ad3e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:41.877945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1d203da0-e8e4-4849-b6db-5981aba44f22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2516fd8fdd429982be5e6495d3ad3e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:41.885029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1083/2131] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:41 2022] DELETE /identity/v3/users/7c2516fd8fdd429982be5e6495d3ad3e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:38:41.889673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc03029-ad8f-438b-92b3-1164e750bc78 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:41.890019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc03029-ad8f-438b-92b3-1164e750bc78 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:41.890331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc03029-ad8f-438b-92b3-1164e750bc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:41.916136 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-5cc03029-ad8f-438b-92b3-1164e750bc78 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:41.917514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5cc03029-ad8f-438b-92b3-1164e750bc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:41.957426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:42.015570 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1049/2132] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:38:42.031624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d203da0-e8e4-4849-b6db-5981aba44f22 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:42.034989 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1d203da0-e8e4-4849-b6db-5981aba44f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:42.099446 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:42.113952 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d203da0-e8e4-4849-b6db-5981aba44f22 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:42.120399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d107776-34eb-40ca-a9c9-9462cc198c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:42.123348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d107776-34eb-40ca-a9c9-9462cc198c56 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:42.123991 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d107776-34eb-40ca-a9c9-9462cc198c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:42.124585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d107776-34eb-40ca-a9c9-9462cc198c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:42.146643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d107776-34eb-40ca-a9c9-9462cc198c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:42.148577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d107776-34eb-40ca-a9c9-9462cc198c56 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:42.163107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1084/2133] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:42.528354 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc03029-ad8f-438b-92b3-1164e750bc78 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:42.530878 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5cc03029-ad8f-438b-92b3-1164e750bc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:42.554287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc03029-ad8f-438b-92b3-1164e750bc78 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:42.565909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7db37a-bdfc-4985-8e17-db19f4c1b036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:42.567256 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7db37a-bdfc-4985-8e17-db19f4c1b036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:42.567256 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7db37a-bdfc-4985-8e17-db19f4c1b036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:42.567256 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7db37a-bdfc-4985-8e17-db19f4c1b036 admin admin] PATH_INFO: `/v3/projects/843586eece33415f8dcc8d5060105ef7` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:42.568509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7db37a-bdfc-4985-8e17-db19f4c1b036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=843586eece33415f8dcc8d5060105ef7)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:42.570682 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7db37a-bdfc-4985-8e17-db19f4c1b036 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:42.663189 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-8a7db37a-bdfc-4985-8e17-db19f4c1b036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '843586eece33415f8dcc8d5060105ef7', 'request_id': 'req-8a7db37a-bdfc-4985-8e17-db19f4c1b036'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:42.676775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-8a7db37a-bdfc-4985-8e17-db19f4c1b036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '843586eece33415f8dcc8d5060105ef7', 'request_id': 'req-8a7db37a-bdfc-4985-8e17-db19f4c1b036'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:42.685057 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-8a7db37a-bdfc-4985-8e17-db19f4c1b036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '843586eece33415f8dcc8d5060105ef7', 'request_id': 'req-8a7db37a-bdfc-4985-8e17-db19f4c1b036'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:42.711013 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-8a7db37a-bdfc-4985-8e17-db19f4c1b036 admin admin] The token cache is being invalidate because project 843586eece33415f8dcc8d5060105ef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:42.712317 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-8a7db37a-bdfc-4985-8e17-db19f4c1b036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:42.713373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-8a7db37a-bdfc-4985-8e17-db19f4c1b036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:42.715858 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1050/2134] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:42 2022] DELETE /identity/v3/projects/843586eece33415f8dcc8d5060105ef7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:42.721377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885fdf7-1a87-42f4-8f86-373472615f90 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:42.721663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885fdf7-1a87-42f4-8f86-373472615f90 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:42.721934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885fdf7-1a87-42f4-8f86-373472615f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:42.742543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5885fdf7-1a87-42f4-8f86-373472615f90 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:42.743928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5885fdf7-1a87-42f4-8f86-373472615f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:42.805077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1085/2135] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:42.813077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eead6c-2d05-44f7-938f-4ef12c5ea40d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:42.813359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eead6c-2d05-44f7-938f-4ef12c5ea40d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:42.813626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eead6c-2d05-44f7-938f-4ef12c5ea40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:42.834326 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-09eead6c-2d05-44f7-938f-4ef12c5ea40d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:42.835553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-09eead6c-2d05-44f7-938f-4ef12c5ea40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:42.844583 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1051/2136] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:42.849043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885fdf7-1a87-42f4-8f86-373472615f90 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:42.859324 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885fdf7-1a87-42f4-8f86-373472615f90 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:42.864092 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51336d-10e6-4e2b-b006-3effec3ca831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:42.864715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51336d-10e6-4e2b-b006-3effec3ca831 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:42.864959 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51336d-10e6-4e2b-b006-3effec3ca831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:42.865262 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51336d-10e6-4e2b-b006-3effec3ca831 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:42.865994 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51336d-10e6-4e2b-b006-3effec3ca831 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:38:42.866204 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51336d-10e6-4e2b-b006-3effec3ca831 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:42.867849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51336d-10e6-4e2b-b006-3effec3ca831 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:42.880718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1086/2137] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:38:42.894922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eead6c-2d05-44f7-938f-4ef12c5ea40d None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:42.906011 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eead6c-2d05-44f7-938f-4ef12c5ea40d None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:42.911908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcd421-34d3-4bd8-b33c-cb76491e1c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:42.912571 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcd421-34d3-4bd8-b33c-cb76491e1c5d admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:42.912817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcd421-34d3-4bd8-b33c-cb76491e1c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:42.913289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcd421-34d3-4bd8-b33c-cb76491e1c5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:42.913910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcd421-34d3-4bd8-b33c-cb76491e1c5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:42.915881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcd421-34d3-4bd8-b33c-cb76491e1c5d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:42.953950 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1052/2138] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:42 2022] POST /identity/v3/projects => generated 397 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:42.959735 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51336d-10e6-4e2b-b006-3effec3ca831 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:42.976375 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51336d-10e6-4e2b-b006-3effec3ca831 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:43.047652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcd421-34d3-4bd8-b33c-cb76491e1c5d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:43.049727 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-32fcd421-34d3-4bd8-b33c-cb76491e1c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:43.066599 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8467d57-a99b-4b32-801b-e638a1bda484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:43.068608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8467d57-a99b-4b32-801b-e638a1bda484 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:43.068952 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8467d57-a99b-4b32-801b-e638a1bda484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:43.069399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8467d57-a99b-4b32-801b-e638a1bda484 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:43.070695 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8467d57-a99b-4b32-801b-e638a1bda484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:43.075264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8467d57-a99b-4b32-801b-e638a1bda484 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:43.124280 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1087/2139] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:42 2022] POST /identity/v3/users => generated 299 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:43.126626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8467d57-a99b-4b32-801b-e638a1bda484 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:43.143236 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcd421-34d3-4bd8-b33c-cb76491e1c5d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:43.148370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:43.149314 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:43.149593 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:43.149871 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] PATH_INFO: `/v3/projects/cfc7d3f0e5104028a60f5a70900b5eb5` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:43.150550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8467d57-a99b-4b32-801b-e638a1bda484 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:43.151192 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfc7d3f0e5104028a60f5a70900b5eb5)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:43.153161 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:43.159186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fd2ab-12c5-4bf0-98bf-e0bb3633a5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:43.160357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fd2ab-12c5-4bf0-98bf-e0bb3633a5da admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:43.160915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fd2ab-12c5-4bf0-98bf-e0bb3633a5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:43.161487 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641fd2ab-12c5-4bf0-98bf-e0bb3633a5da admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:43.163066 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641fd2ab-12c5-4bf0-98bf-e0bb3633a5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:43.166292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641fd2ab-12c5-4bf0-98bf-e0bb3633a5da admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:43.191266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1088/2140] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:38:43.196590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fd2ab-12c5-4bf0-98bf-e0bb3633a5da admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:43.209803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:43.213205 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641fd2ab-12c5-4bf0-98bf-e0bb3633a5da admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:43.228310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfc7d3f0e5104028a60f5a70900b5eb5', 'request_id': 'req-10f399f1-01ed-42ff-9a4d-da066d182ac0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:43.238177 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfc7d3f0e5104028a60f5a70900b5eb5', 'request_id': 'req-10f399f1-01ed-42ff-9a4d-da066d182ac0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:43.244611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfc7d3f0e5104028a60f5a70900b5eb5', 'request_id': 'req-10f399f1-01ed-42ff-9a4d-da066d182ac0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:43.257816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] The token cache is being invalidate because project cfc7d3f0e5104028a60f5a70900b5eb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:43.258188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:43.258793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:43.261251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1053/2141] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:43 2022] DELETE /identity/v3/projects/cfc7d3f0e5104028a60f5a70900b5eb5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:38:43.265613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:43.267474 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:43.289505 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:43.308046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953c780-cda4-41b1-a62c-d1052c54ff18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:43.310940 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953c780-cda4-41b1-a62c-d1052c54ff18 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:43.312685 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953c780-cda4-41b1-a62c-d1052c54ff18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:43.312811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953c780-cda4-41b1-a62c-d1052c54ff18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:43.313712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953c780-cda4-41b1-a62c-d1052c54ff18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:43.316766 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953c780-cda4-41b1-a62c-d1052c54ff18 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:43.342056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1089/2142] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:43.342727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953c780-cda4-41b1-a62c-d1052c54ff18 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:43.345190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9953c780-cda4-41b1-a62c-d1052c54ff18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:43.365324 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953c780-cda4-41b1-a62c-d1052c54ff18 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:43.376393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f388798-7e51-42a1-a8c5-8d65497f1861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:43.377853 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f388798-7e51-42a1-a8c5-8d65497f1861 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:43.378257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f388798-7e51-42a1-a8c5-8d65497f1861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:43.378598 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f388798-7e51-42a1-a8c5-8d65497f1861 admin admin] PATH_INFO: `/v3/projects/4064442f79ee488da55703a4ef764a82/users/5ddde388a4c4478e976d81305ba6b58e/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:43.388676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f399f1-01ed-42ff-9a4d-da066d182ac0 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:43.395143 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1d82f-dfcd-4b2f-a15a-fec450ee843a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:43.396007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1d82f-dfcd-4b2f-a15a-fec450ee843a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:43.396332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1d82f-dfcd-4b2f-a15a-fec450ee843a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:43.396883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1d82f-dfcd-4b2f-a15a-fec450ee843a admin admin] PATH_INFO: `/v3/users/8f166fa29e7d4864af80c8f21d9c3454` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:43.399307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1d82f-dfcd-4b2f-a15a-fec450ee843a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f166fa29e7d4864af80c8f21d9c3454)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:43.402365 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f388798-7e51-42a1-a8c5-8d65497f1861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4064442f79ee488da55703a4ef764a82, user_id=5ddde388a4c4478e976d81305ba6b58e, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:43.402797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1d82f-dfcd-4b2f-a15a-fec450ee843a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:43.407294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f388798-7e51-42a1-a8c5-8d65497f1861 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:43.430310 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1090/2143] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:43 2022] PUT /identity/v3/projects/4064442f79ee488da55703a4ef764a82/users/5ddde388a4c4478e976d81305ba6b58e/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:43.442389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0d170-d5fb-4faf-a1df-37d0536247ff None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:43.442547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0d170-d5fb-4faf-a1df-37d0536247ff None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:43.442727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0d170-d5fb-4faf-a1df-37d0536247ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:43.511178 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-def0d170-d5fb-4faf-a1df-37d0536247ff None None] MFA Rules not processed for user `5ddde388a4c4478e976d81305ba6b58e`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:43.512453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-def0d170-d5fb-4faf-a1df-37d0536247ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:43.537620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-02a1d82f-dfcd-4b2f-a15a-fec450ee843a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f166fa29e7d4864af80c8f21d9c3454'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:43.601419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-02a1d82f-dfcd-4b2f-a15a-fec450ee843a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f166fa29e7d4864af80c8f21d9c3454'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:43.631331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-02a1d82f-dfcd-4b2f-a15a-fec450ee843a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f166fa29e7d4864af80c8f21d9c3454'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:43.653557 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-02a1d82f-dfcd-4b2f-a15a-fec450ee843a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f166fa29e7d4864af80c8f21d9c3454'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:43.654640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-02a1d82f-dfcd-4b2f-a15a-fec450ee843a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f166fa29e7d4864af80c8f21d9c3454'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:43.673823 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1054/2144] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:43 2022] DELETE /identity/v3/users/8f166fa29e7d4864af80c8f21d9c3454 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:43.686699 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1091/2145] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:43 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:38:43.700638 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1d82f-dfcd-4b2f-a15a-fec450ee843a admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:43.702834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-02a1d82f-dfcd-4b2f-a15a-fec450ee843a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:43.790660 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:43.798790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d007772-9e46-4bb2-bafe-e663bf09e5c2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:43.799109 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d007772-9e46-4bb2-bafe-e663bf09e5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:43.799437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d007772-9e46-4bb2-bafe-e663bf09e5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:43.816001 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1d82f-dfcd-4b2f-a15a-fec450ee843a admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:43.821263 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a5a8f-ad16-4a21-b435-d712dc10a80b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:43.822006 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a5a8f-ad16-4a21-b435-d712dc10a80b service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:43.822229 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7d007772-9e46-4bb2-bafe-e663bf09e5c2 None None] MFA Rules not processed for user `52760b6736e14ccd8df8aca0015cb292`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:43.822341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a5a8f-ad16-4a21-b435-d712dc10a80b service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:43.822680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a5a8f-ad16-4a21-b435-d712dc10a80b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:43.823449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7d007772-9e46-4bb2-bafe-e663bf09e5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:43.890142 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1092/2146] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:43.906114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a5a8f-ad16-4a21-b435-d712dc10a80b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:43.907717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a5a8f-ad16-4a21-b435-d712dc10a80b service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:43.918894 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1055/2147] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:43 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:44.474623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d007772-9e46-4bb2-bafe-e663bf09e5c2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:44.476126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7d007772-9e46-4bb2-bafe-e663bf09e5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:44.623816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d007772-9e46-4bb2-bafe-e663bf09e5c2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:44.632728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c55e95-5e6f-4e3b-a844-533c6eb6b934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:44.633894 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c55e95-5e6f-4e3b-a844-533c6eb6b934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:44.634449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c55e95-5e6f-4e3b-a844-533c6eb6b934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:44.634998 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c55e95-5e6f-4e3b-a844-533c6eb6b934 admin admin] PATH_INFO: `/v3/projects/1dc197b087c843e38552f19cdf95b651` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:44.637037 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c55e95-5e6f-4e3b-a844-533c6eb6b934 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dc197b087c843e38552f19cdf95b651)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:44.641248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c55e95-5e6f-4e3b-a844-533c6eb6b934 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:44.644343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:44.781531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22c55e95-5e6f-4e3b-a844-533c6eb6b934 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dc197b087c843e38552f19cdf95b651', 'request_id': 'req-22c55e95-5e6f-4e3b-a844-533c6eb6b934'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:44.792410 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22c55e95-5e6f-4e3b-a844-533c6eb6b934 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dc197b087c843e38552f19cdf95b651', 'request_id': 'req-22c55e95-5e6f-4e3b-a844-533c6eb6b934'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:44.813615 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22c55e95-5e6f-4e3b-a844-533c6eb6b934 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dc197b087c843e38552f19cdf95b651', 'request_id': 'req-22c55e95-5e6f-4e3b-a844-533c6eb6b934'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:44.846654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22c55e95-5e6f-4e3b-a844-533c6eb6b934 admin admin] The token cache is being invalidate because project 1dc197b087c843e38552f19cdf95b651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:44.847033 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22c55e95-5e6f-4e3b-a844-533c6eb6b934 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:44.847844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22c55e95-5e6f-4e3b-a844-533c6eb6b934 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:44.852025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1093/2148] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:44 2022] DELETE /identity/v3/projects/1dc197b087c843e38552f19cdf95b651 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:44.857184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef74a0-e61d-423f-b8a5-c7b8ed4254ca None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:44.857637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef74a0-e61d-423f-b8a5-c7b8ed4254ca None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:44.857889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef74a0-e61d-423f-b8a5-c7b8ed4254ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:44.891686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-7aef74a0-e61d-423f-b8a5-c7b8ed4254ca None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:44.893196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7aef74a0-e61d-423f-b8a5-c7b8ed4254ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:44.985151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1056/2149] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:44.995118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c1177-82b0-4f57-85f7-e9d7b4cedbdb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:44.995756 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c1177-82b0-4f57-85f7-e9d7b4cedbdb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:44.996070 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c1177-82b0-4f57-85f7-e9d7b4cedbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:45.033375 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5b9c1177-82b0-4f57-85f7-e9d7b4cedbdb None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:45.034833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5b9c1177-82b0-4f57-85f7-e9d7b4cedbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:45.047697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1094/2150] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:45.052007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef74a0-e61d-423f-b8a5-c7b8ed4254ca None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:45.083529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef74a0-e61d-423f-b8a5-c7b8ed4254ca None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:45.100630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0fa27-b0c2-4122-adaf-02c96d043861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:45.101484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0fa27-b0c2-4122-adaf-02c96d043861 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:45.101790 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0fa27-b0c2-4122-adaf-02c96d043861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:45.102832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0fa27-b0c2-4122-adaf-02c96d043861 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:45.104640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0fa27-b0c2-4122-adaf-02c96d043861 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:38:45.106130 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0fa27-b0c2-4122-adaf-02c96d043861 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:45.111382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0fa27-b0c2-4122-adaf-02c96d043861 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:45.143531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1057/2151] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:38:45.148559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c1177-82b0-4f57-85f7-e9d7b4cedbdb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:45.165440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c1177-82b0-4f57-85f7-e9d7b4cedbdb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:45.175212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606159b-bfb1-48fc-88d6-bd08339d2a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:45.177027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606159b-bfb1-48fc-88d6-bd08339d2a19 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:45.177402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606159b-bfb1-48fc-88d6-bd08339d2a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:45.177691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606159b-bfb1-48fc-88d6-bd08339d2a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:45.179231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606159b-bfb1-48fc-88d6-bd08339d2a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:45.182332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606159b-bfb1-48fc-88d6-bd08339d2a19 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:45.255371 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1095/2152] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:45 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:38:45.257883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0fa27-b0c2-4122-adaf-02c96d043861 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:45.294098 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0fa27-b0c2-4122-adaf-02c96d043861 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:45.415836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a55685-b75c-4c60-ba8e-c3b7270b0d6c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:45.416349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a55685-b75c-4c60-ba8e-c3b7270b0d6c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:45.416631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a55685-b75c-4c60-ba8e-c3b7270b0d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:45.444283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ec778-fcfb-4dac-a8b7-9054e939d7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:45.447627 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec778-fcfb-4dac-a8b7-9054e939d7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:45.449041 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec778-fcfb-4dac-a8b7-9054e939d7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:45.449335 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec778-fcfb-4dac-a8b7-9054e939d7a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:45.454420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ec778-fcfb-4dac-a8b7-9054e939d7a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:45.456515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ec778-fcfb-4dac-a8b7-9054e939d7a3 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:45.470924 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-39a55685-b75c-4c60-ba8e-c3b7270b0d6c None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:45.473971 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-39a55685-b75c-4c60-ba8e-c3b7270b0d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:45.505140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1058/2153] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:45 2022] POST /identity/v3/users => generated 296 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:45.510821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ec778-fcfb-4dac-a8b7-9054e939d7a3 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:45.526421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ec778-fcfb-4dac-a8b7-9054e939d7a3 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:45.533465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91648045-a9a5-42cd-bcc0-75d28e6dc267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:45.535052 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91648045-a9a5-42cd-bcc0-75d28e6dc267 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:45.535336 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91648045-a9a5-42cd-bcc0-75d28e6dc267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:45.536223 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91648045-a9a5-42cd-bcc0-75d28e6dc267 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:45.537821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91648045-a9a5-42cd-bcc0-75d28e6dc267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:45.544557 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91648045-a9a5-42cd-bcc0-75d28e6dc267 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:45.553798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:45.575956 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1059/2154] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:38:45.580881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91648045-a9a5-42cd-bcc0-75d28e6dc267 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:45.613386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1096/2155] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:38:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:45.619678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91648045-a9a5-42cd-bcc0-75d28e6dc267 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:45.626194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cd931-6d00-4b3a-a477-7b3548335515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:45.626986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cd931-6d00-4b3a-a477-7b3548335515 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:45.627324 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cd931-6d00-4b3a-a477-7b3548335515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:45.627801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96cd931-6d00-4b3a-a477-7b3548335515 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:45.629045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96cd931-6d00-4b3a-a477-7b3548335515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:45.630785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96cd931-6d00-4b3a-a477-7b3548335515 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:45.649662 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1060/2156] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:45.654142 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a55685-b75c-4c60-ba8e-c3b7270b0d6c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:45.683326 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a55685-b75c-4c60-ba8e-c3b7270b0d6c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:45.696804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cd931-6d00-4b3a-a477-7b3548335515 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:45.702559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a96cd931-6d00-4b3a-a477-7b3548335515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:45.718004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a3309-e48f-41fc-ad3a-a54ec0d223f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:45.722489 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a3309-e48f-41fc-ad3a-a54ec0d223f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:45.722787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a3309-e48f-41fc-ad3a-a54ec0d223f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:45.723061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a3309-e48f-41fc-ad3a-a54ec0d223f7 admin admin] PATH_INFO: `/v3/projects/54ac2f0a43324c2eb43ba3e70e7a9b4a/users/2d5155b3ad8949cf90f6367608ceaa1e/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:45.772451 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a3309-e48f-41fc-ad3a-a54ec0d223f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ac2f0a43324c2eb43ba3e70e7a9b4a, user_id=2d5155b3ad8949cf90f6367608ceaa1e, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:45.780582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a3309-e48f-41fc-ad3a-a54ec0d223f7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:45.841539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96cd931-6d00-4b3a-a477-7b3548335515 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:45.855377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1097/2157] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:45 2022] PUT /identity/v3/projects/54ac2f0a43324c2eb43ba3e70e7a9b4a/users/2d5155b3ad8949cf90f6367608ceaa1e/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:45.877057 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d02d3-ea01-42b9-ac66-dff5dbcf8f9b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:45.877335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d02d3-ea01-42b9-ac66-dff5dbcf8f9b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:45.877660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d02d3-ea01-42b9-ac66-dff5dbcf8f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:45.976167 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963194a1-f62d-418a-a8f6-20014598fd75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:45.976996 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963194a1-f62d-418a-a8f6-20014598fd75 service swift] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:45.977516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963194a1-f62d-418a-a8f6-20014598fd75 service swift] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:45.977848 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963194a1-f62d-418a-a8f6-20014598fd75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:46.001102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-465d02d3-ea01-42b9-ac66-dff5dbcf8f9b None None] MFA Rules not processed for user `2d5155b3ad8949cf90f6367608ceaa1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:46.004314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-465d02d3-ea01-42b9-ac66-dff5dbcf8f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:46.074767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:46.076765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963194a1-f62d-418a-a8f6-20014598fd75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:46.084051 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963194a1-f62d-418a-a8f6-20014598fd75 service swift] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:46.101037 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1061/2158] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:38:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:38:46.134946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1098/2159] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:38:46.142562 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963194a1-f62d-418a-a8f6-20014598fd75 service swift] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:46.255865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:46.334074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963194a1-f62d-418a-a8f6-20014598fd75 service swift] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:46.358995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda39238-231f-43b0-9683-2a44683c0182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:46.359640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda39238-231f-43b0-9683-2a44683c0182 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:46.359941 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda39238-231f-43b0-9683-2a44683c0182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:46.360218 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda39238-231f-43b0-9683-2a44683c0182 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:46.361142 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda39238-231f-43b0-9683-2a44683c0182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:46.370612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda39238-231f-43b0-9683-2a44683c0182 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:46.437663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1062/2160] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:46 2022] POST /identity/v3/projects => generated 391 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:46.438462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d02d3-ea01-42b9-ac66-dff5dbcf8f9b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:46.482941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d02d3-ea01-42b9-ac66-dff5dbcf8f9b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:46.636789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285dc5b-9633-48b5-aa83-9c24c8270969 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:46.638279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285dc5b-9633-48b5-aa83-9c24c8270969 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:46.638849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285dc5b-9633-48b5-aa83-9c24c8270969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:46.642748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7662c6-6d30-4da2-a5f3-3f08b9d71932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:46.643410 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7662c6-6d30-4da2-a5f3-3f08b9d71932 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:46.645979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7662c6-6d30-4da2-a5f3-3f08b9d71932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:46.646248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7662c6-6d30-4da2-a5f3-3f08b9d71932 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:46.648380 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7662c6-6d30-4da2-a5f3-3f08b9d71932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:46.653962 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7662c6-6d30-4da2-a5f3-3f08b9d71932 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:46.696914 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-d285dc5b-9633-48b5-aa83-9c24c8270969 None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:46.699643 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d285dc5b-9633-48b5-aa83-9c24c8270969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:46.716755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1099/2161] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:46 2022] POST /identity/v3/users => generated 302 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:38:46.720823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7662c6-6d30-4da2-a5f3-3f08b9d71932 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:46.743374 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7662c6-6d30-4da2-a5f3-3f08b9d71932 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:46.755315 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e95502-33f3-4c0e-8ff4-b2faef1b3a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:46.761765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e95502-33f3-4c0e-8ff4-b2faef1b3a9c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:46.762071 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e95502-33f3-4c0e-8ff4-b2faef1b3a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:46.762341 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e95502-33f3-4c0e-8ff4-b2faef1b3a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:46.763275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e95502-33f3-4c0e-8ff4-b2faef1b3a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:46.765406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e95502-33f3-4c0e-8ff4-b2faef1b3a9c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:46.781020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1100/2162] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:46.785927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e95502-33f3-4c0e-8ff4-b2faef1b3a9c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:46.800643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e95502-33f3-4c0e-8ff4-b2faef1b3a9c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:46.807685 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea4d91-5db9-447d-8204-c392f0e83e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:46.808493 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea4d91-5db9-447d-8204-c392f0e83e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:46.808776 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea4d91-5db9-447d-8204-c392f0e83e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:46.809052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea4d91-5db9-447d-8204-c392f0e83e23 admin admin] PATH_INFO: `/v3/projects/2a2da1a86b5a4751b2acf6791ece0167/users/2422d9e1db3a44fea94d2de4d6b81d48/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:46.832157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea4d91-5db9-447d-8204-c392f0e83e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2da1a86b5a4751b2acf6791ece0167, user_id=2422d9e1db3a44fea94d2de4d6b81d48, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:46.836233 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea4d91-5db9-447d-8204-c392f0e83e23 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:46.857942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1063/2163] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:38:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:38:46.864296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1101/2164] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:46 2022] PUT /identity/v3/projects/2a2da1a86b5a4751b2acf6791ece0167/users/2422d9e1db3a44fea94d2de4d6b81d48/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:46.869255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285dc5b-9633-48b5-aa83-9c24c8270969 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:46.888413 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285dc5b-9633-48b5-aa83-9c24c8270969 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:46.902865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea4d91-5db9-447d-8204-c392f0e83e23 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:47.042484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b1946-0827-45c0-8649-bb223d170ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:47.043090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b1946-0827-45c0-8649-bb223d170ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:47.043376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b1946-0827-45c0-8649-bb223d170ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:47.043638 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b1946-0827-45c0-8649-bb223d170ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:47.045218 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b1946-0827-45c0-8649-bb223d170ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:47.047096 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b1946-0827-45c0-8649-bb223d170ce8 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:47.060726 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1064/2165] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:38:47.066540 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b1946-0827-45c0-8649-bb223d170ce8 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:47.067012 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea4d91-5db9-447d-8204-c392f0e83e23 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:47.078199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc8a40-38fd-460d-bc16-439a3fe6d3dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:47.078991 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc8a40-38fd-460d-bc16-439a3fe6d3dd service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:47.079818 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc8a40-38fd-460d-bc16-439a3fe6d3dd service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:47.080145 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc8a40-38fd-460d-bc16-439a3fe6d3dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:47.084181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b1946-0827-45c0-8649-bb223d170ce8 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:47.089664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21053cc-9dd5-4255-9def-f0f83a42de84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:47.090456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21053cc-9dd5-4255-9def-f0f83a42de84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:47.091343 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21053cc-9dd5-4255-9def-f0f83a42de84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:47.091343 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21053cc-9dd5-4255-9def-f0f83a42de84 admin admin] PATH_INFO: `/v3/projects/2a2da1a86b5a4751b2acf6791ece0167/users/2422d9e1db3a44fea94d2de4d6b81d48/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:47.094695 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21053cc-9dd5-4255-9def-f0f83a42de84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2da1a86b5a4751b2acf6791ece0167, user_id=2422d9e1db3a44fea94d2de4d6b81d48, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:47.097982 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21053cc-9dd5-4255-9def-f0f83a42de84 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:47.125349 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1065/2166] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:38:47 2022] PUT /identity/v3/projects/2a2da1a86b5a4751b2acf6791ece0167/users/2422d9e1db3a44fea94d2de4d6b81d48/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:47.132358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd91446-a080-4fd9-aac6-50a871a2bd24 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:47.132819 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd91446-a080-4fd9-aac6-50a871a2bd24 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:47.133202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd91446-a080-4fd9-aac6-50a871a2bd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:47.192614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:47.262449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-7bd91446-a080-4fd9-aac6-50a871a2bd24 None None] MFA Rules not processed for user `2422d9e1db3a44fea94d2de4d6b81d48`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:47.266202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7bd91446-a080-4fd9-aac6-50a871a2bd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:47.321850 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc8a40-38fd-460d-bc16-439a3fe6d3dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:47.324146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc8a40-38fd-460d-bc16-439a3fe6d3dd service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:47.356020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1102/2167] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:38:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 16:38:47.395755 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:47.608886 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1066/2168] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:47 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:47.635075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc8a40-38fd-460d-bc16-439a3fe6d3dd service swift] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:47.636872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8abc8a40-38fd-460d-bc16-439a3fe6d3dd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:47.734382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c480843-f7b0-4af3-b592-2c675251973d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:47.734702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c480843-f7b0-4af3-b592-2c675251973d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:47.736470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c480843-f7b0-4af3-b592-2c675251973d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:47.807389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc8a40-38fd-460d-bc16-439a3fe6d3dd service swift] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:47.819282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-1c480843-f7b0-4af3-b592-2c675251973d None None] MFA Rules not processed for user `3a9b222396b1438f9fa0af0e197a9293`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:47.820568 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1c480843-f7b0-4af3-b592-2c675251973d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:47.931399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0d019-e2aa-492c-a92c-1d2862ab57be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:47.932132 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0d019-e2aa-492c-a92c-1d2862ab57be service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:47.932410 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0d019-e2aa-492c-a92c-1d2862ab57be service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:47.932678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0d019-e2aa-492c-a92c-1d2862ab57be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:47.954410 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0d019-e2aa-492c-a92c-1d2862ab57be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:47.956951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0d019-e2aa-492c-a92c-1d2862ab57be service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:47.965639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1067/2169] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:47.973638 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1103/2170] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:47 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:48.109487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c480843-f7b0-4af3-b592-2c675251973d None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:48.129061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c480843-f7b0-4af3-b592-2c675251973d None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:48.140668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569a026-2dff-457c-bd6a-27e26324d808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:48.141452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569a026-2dff-457c-bd6a-27e26324d808 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:48.141733 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569a026-2dff-457c-bd6a-27e26324d808 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:48.142002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569a026-2dff-457c-bd6a-27e26324d808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:48.274205 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569a026-2dff-457c-bd6a-27e26324d808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:48.276792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569a026-2dff-457c-bd6a-27e26324d808 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:48.306899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1068/2171] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:48 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:48.688625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16fa13-8f11-42df-9ebc-0a1585549cdf None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:48.689730 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16fa13-8f11-42df-9ebc-0a1585549cdf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:48.690170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16fa13-8f11-42df-9ebc-0a1585549cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:48.720867 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5b16fa13-8f11-42df-9ebc-0a1585549cdf None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:48.722694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5b16fa13-8f11-42df-9ebc-0a1585549cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:48.826654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1104/2172] 10.222.0.62 () {58 vars in 967 bytes} [Thu Sep 1 16:38:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:48.887596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569a026-2dff-457c-bd6a-27e26324d808 service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:49.013646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569a026-2dff-457c-bd6a-27e26324d808 service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:49.021114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbcc13-2166-47f7-b8cb-06bb57a3a105 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:49.022081 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbcc13-2166-47f7-b8cb-06bb57a3a105 service swift] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:49.022538 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbcc13-2166-47f7-b8cb-06bb57a3a105 service swift] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:49.023047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbcc13-2166-47f7-b8cb-06bb57a3a105 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:49.036706 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbcc13-2166-47f7-b8cb-06bb57a3a105 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:49.039020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bbcc13-2166-47f7-b8cb-06bb57a3a105 service swift] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:49.053088 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1069/2173] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:38:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:38:49.663635 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3424f37-b10b-410c-ac79-c3599b170502 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:49.664233 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3424f37-b10b-410c-ac79-c3599b170502 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:49.664692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3424f37-b10b-410c-ac79-c3599b170502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:49.704876 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a3424f37-b10b-410c-ac79-c3599b170502 None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:49.719535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a3424f37-b10b-410c-ac79-c3599b170502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:49.735912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1105/2174] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:38:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:49.744958 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbcc13-2166-47f7-b8cb-06bb57a3a105 service swift] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:49.760914 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbcc13-2166-47f7-b8cb-06bb57a3a105 service swift] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:49.769976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:49.771403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf125c7-a6d6-4ab5-81af-9e171e9f7003 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:49.772522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf125c7-a6d6-4ab5-81af-9e171e9f7003 service swift] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:49.772910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf125c7-a6d6-4ab5-81af-9e171e9f7003 service swift] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:49.773408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf125c7-a6d6-4ab5-81af-9e171e9f7003 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:49.789925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf125c7-a6d6-4ab5-81af-9e171e9f7003 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:49.791755 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf125c7-a6d6-4ab5-81af-9e171e9f7003 service swift] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:49.810223 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1070/2175] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:38:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:38:50.610358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69068ee9-4de5-450e-8b3b-1173cea4bc3a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:50.610923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69068ee9-4de5-450e-8b3b-1173cea4bc3a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:50.610923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69068ee9-4de5-450e-8b3b-1173cea4bc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:50.639864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-69068ee9-4de5-450e-8b3b-1173cea4bc3a None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:50.642296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-69068ee9-4de5-450e-8b3b-1173cea4bc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:50.653242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1106/2176] 10.222.0.62 () {58 vars in 967 bytes} [Thu Sep 1 16:38:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:50.666493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf125c7-a6d6-4ab5-81af-9e171e9f7003 service swift] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:50.681975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf125c7-a6d6-4ab5-81af-9e171e9f7003 service swift] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:50.703865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4c088-da4f-4213-a00e-3adc4ba516e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:50.706096 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4c088-da4f-4213-a00e-3adc4ba516e9 service swift] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:50.708732 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4c088-da4f-4213-a00e-3adc4ba516e9 service swift] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:50.709246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4c088-da4f-4213-a00e-3adc4ba516e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:50.727865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4c088-da4f-4213-a00e-3adc4ba516e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:50.731615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4c088-da4f-4213-a00e-3adc4ba516e9 service swift] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:50.753783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1071/2177] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:38:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:38:52.023746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69068ee9-4de5-450e-8b3b-1173cea4bc3a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:52.025975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-69068ee9-4de5-450e-8b3b-1173cea4bc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:52.083939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:52.158375 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69068ee9-4de5-450e-8b3b-1173cea4bc3a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:52.165671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb2dc3-4fdd-4451-b671-ff452cf7cf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:52.166750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb2dc3-4fdd-4451-b671-ff452cf7cf7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:52.167034 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb2dc3-4fdd-4451-b671-ff452cf7cf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:52.167307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb2dc3-4fdd-4451-b671-ff452cf7cf7a admin admin] PATH_INFO: `/v3/users/5ddde388a4c4478e976d81305ba6b58e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:52.169449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb2dc3-4fdd-4451-b671-ff452cf7cf7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ddde388a4c4478e976d81305ba6b58e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:52.172531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb2dc3-4fdd-4451-b671-ff452cf7cf7a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:52.260845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3ceb2dc3-4fdd-4451-b671-ff452cf7cf7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ddde388a4c4478e976d81305ba6b58e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.286808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3ceb2dc3-4fdd-4451-b671-ff452cf7cf7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ddde388a4c4478e976d81305ba6b58e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.288008 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3ceb2dc3-4fdd-4451-b671-ff452cf7cf7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ddde388a4c4478e976d81305ba6b58e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.333708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3ceb2dc3-4fdd-4451-b671-ff452cf7cf7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ddde388a4c4478e976d81305ba6b58e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.347173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3ceb2dc3-4fdd-4451-b671-ff452cf7cf7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ddde388a4c4478e976d81305ba6b58e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:52.360624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1107/2178] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:52 2022] DELETE /identity/v3/users/5ddde388a4c4478e976d81305ba6b58e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:52.365964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd97db-aafc-4025-8a5e-2c031a5fedad None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:52.366576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd97db-aafc-4025-8a5e-2c031a5fedad None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:52.366576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd97db-aafc-4025-8a5e-2c031a5fedad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:52.411717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-fffd97db-aafc-4025-8a5e-2c031a5fedad None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:52.413022 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-fffd97db-aafc-4025-8a5e-2c031a5fedad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:52.509120 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1072/2179] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:52.526822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb2dc3-4fdd-4451-b671-ff452cf7cf7a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:52.533033 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3ceb2dc3-4fdd-4451-b671-ff452cf7cf7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:52.688709 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb2dc3-4fdd-4451-b671-ff452cf7cf7a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:52.693183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d5976-b9ba-408e-944e-090ad58f0fc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:52.694362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d5976-b9ba-408e-944e-090ad58f0fc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:52.694707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d5976-b9ba-408e-944e-090ad58f0fc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:52.695032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d5976-b9ba-408e-944e-090ad58f0fc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:52.733401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d5976-b9ba-408e-944e-090ad58f0fc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:52.735332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d5976-b9ba-408e-944e-090ad58f0fc5 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:52.782521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1108/2180] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:53.042506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd97db-aafc-4025-8a5e-2c031a5fedad None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:53.054981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd97db-aafc-4025-8a5e-2c031a5fedad None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:53.069577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73bbad-e48c-4dc3-b45e-8ba65a60995e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:53.070112 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73bbad-e48c-4dc3-b45e-8ba65a60995e service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:53.070389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73bbad-e48c-4dc3-b45e-8ba65a60995e service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:53.070658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73bbad-e48c-4dc3-b45e-8ba65a60995e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:53.082718 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6e73bbad-e48c-4dc3-b45e-8ba65a60995e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:53.295095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73bbad-e48c-4dc3-b45e-8ba65a60995e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:53.297331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73bbad-e48c-4dc3-b45e-8ba65a60995e service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:53.324614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1073/2181] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:53.460660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d5976-b9ba-408e-944e-090ad58f0fc5 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:53.463067 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e62d5976-b9ba-408e-944e-090ad58f0fc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:53.490337 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d5976-b9ba-408e-944e-090ad58f0fc5 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:53.513568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacca04-0a2d-402c-9084-83849a1e4a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:53.514849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacca04-0a2d-402c-9084-83849a1e4a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:53.515477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacca04-0a2d-402c-9084-83849a1e4a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:53.515949 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacca04-0a2d-402c-9084-83849a1e4a7e admin admin] PATH_INFO: `/v3/users/2d5155b3ad8949cf90f6367608ceaa1e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:53.523429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacca04-0a2d-402c-9084-83849a1e4a7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d5155b3ad8949cf90f6367608ceaa1e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:53.526571 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacca04-0a2d-402c-9084-83849a1e4a7e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:53.718033 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:53.760688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2aacca04-0a2d-402c-9084-83849a1e4a7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d5155b3ad8949cf90f6367608ceaa1e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.781871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2aacca04-0a2d-402c-9084-83849a1e4a7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d5155b3ad8949cf90f6367608ceaa1e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.782630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2aacca04-0a2d-402c-9084-83849a1e4a7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d5155b3ad8949cf90f6367608ceaa1e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.827833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2aacca04-0a2d-402c-9084-83849a1e4a7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d5155b3ad8949cf90f6367608ceaa1e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.847331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2aacca04-0a2d-402c-9084-83849a1e4a7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d5155b3ad8949cf90f6367608ceaa1e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:53.856606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1109/2182] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:53 2022] DELETE /identity/v3/users/2d5155b3ad8949cf90f6367608ceaa1e => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:53.868755 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d544-8435-4b96-ab03-977886eeaf6c None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:53.869430 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d544-8435-4b96-ab03-977886eeaf6c None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:53.870129 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d544-8435-4b96-ab03-977886eeaf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:53.922561 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacca04-0a2d-402c-9084-83849a1e4a7e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:53.925301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2aacca04-0a2d-402c-9084-83849a1e4a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:53.937331 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-83e2d544-8435-4b96-ab03-977886eeaf6c None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:53.941788 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-83e2d544-8435-4b96-ab03-977886eeaf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:54.205901 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1074/2183] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:54.247859 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacca04-0a2d-402c-9084-83849a1e4a7e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:54.258226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d544-8435-4b96-ab03-977886eeaf6c None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:54.259536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:54.259710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-83e2d544-8435-4b96-ab03-977886eeaf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:54.260586 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:54.260956 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:54.261315 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] PATH_INFO: `/v3/projects/4064442f79ee488da55703a4ef764a82` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:54.263601 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4064442f79ee488da55703a4ef764a82)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:54.272000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:54.285234 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:54.421996 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d544-8435-4b96-ab03-977886eeaf6c None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:54.464080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4064442f79ee488da55703a4ef764a82', 'request_id': 'req-adb28a51-d867-4946-a6c7-93cc0a1886b3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:54.486239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4064442f79ee488da55703a4ef764a82', 'request_id': 'req-adb28a51-d867-4946-a6c7-93cc0a1886b3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:54.498504 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4064442f79ee488da55703a4ef764a82', 'request_id': 'req-adb28a51-d867-4946-a6c7-93cc0a1886b3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:54.535473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] The token cache is being invalidate because project 4064442f79ee488da55703a4ef764a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:54.538217 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:54.540601 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:54.545078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1110/2184] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:53 2022] DELETE /identity/v3/projects/4064442f79ee488da55703a4ef764a82 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:54.554059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:54.558371 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:54.628161 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8fef6-5838-418c-a66d-aa31231fb9d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:54.631291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8fef6-5838-418c-a66d-aa31231fb9d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:54.633790 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8fef6-5838-418c-a66d-aa31231fb9d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:54.635532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8fef6-5838-418c-a66d-aa31231fb9d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:54.643082 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-40e8fef6-5838-418c-a66d-aa31231fb9d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:54.786048 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:54.900351 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb28a51-d867-4946-a6c7-93cc0a1886b3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:54.912386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36df237-d186-4a8b-8b3c-24ff8c2f0889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:54.912826 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36df237-d186-4a8b-8b3c-24ff8c2f0889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:54.913118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36df237-d186-4a8b-8b3c-24ff8c2f0889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:54.913393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36df237-d186-4a8b-8b3c-24ff8c2f0889 admin admin] PATH_INFO: `/v3/users/535e932c54904b83bfc19fd68df95116` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:54.917957 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36df237-d186-4a8b-8b3c-24ff8c2f0889 admin admin] RBAC: Authorizing `identity:delete_user(user_id=535e932c54904b83bfc19fd68df95116)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:54.920606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36df237-d186-4a8b-8b3c-24ff8c2f0889 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:54.964889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8fef6-5838-418c-a66d-aa31231fb9d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:54.972167 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8fef6-5838-418c-a66d-aa31231fb9d2 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:55.024794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1075/2185] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:38:55.145934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c36df237-d186-4a8b-8b3c-24ff8c2f0889 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '535e932c54904b83bfc19fd68df95116'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:55.162186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c36df237-d186-4a8b-8b3c-24ff8c2f0889 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '535e932c54904b83bfc19fd68df95116'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:55.162804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c36df237-d186-4a8b-8b3c-24ff8c2f0889 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '535e932c54904b83bfc19fd68df95116'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:55.208827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c36df237-d186-4a8b-8b3c-24ff8c2f0889 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '535e932c54904b83bfc19fd68df95116'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:55.235805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c36df237-d186-4a8b-8b3c-24ff8c2f0889 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '535e932c54904b83bfc19fd68df95116'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:55.259558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1111/2186] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:54 2022] DELETE /identity/v3/users/535e932c54904b83bfc19fd68df95116 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:55.264935 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0587b-0804-4fbd-91bd-dacf41eaba09 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:55.265208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0587b-0804-4fbd-91bd-dacf41eaba09 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:55.265531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0587b-0804-4fbd-91bd-dacf41eaba09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:55.321625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-7dd0587b-0804-4fbd-91bd-dacf41eaba09 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:55.323377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7dd0587b-0804-4fbd-91bd-dacf41eaba09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:55.494758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1076/2187] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:55.511765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36df237-d186-4a8b-8b3c-24ff8c2f0889 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:55.514917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c36df237-d186-4a8b-8b3c-24ff8c2f0889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:55.694778 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36df237-d186-4a8b-8b3c-24ff8c2f0889 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:55.698993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3621e87-2c63-46d9-9bd0-5cb206a7aeb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:55.699614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3621e87-2c63-46d9-9bd0-5cb206a7aeb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:55.700057 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3621e87-2c63-46d9-9bd0-5cb206a7aeb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:55.700336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3621e87-2c63-46d9-9bd0-5cb206a7aeb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:55.728029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3621e87-2c63-46d9-9bd0-5cb206a7aeb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:55.729492 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3621e87-2c63-46d9-9bd0-5cb206a7aeb3 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:55.747530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1112/2188] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:57.037256 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0587b-0804-4fbd-91bd-dacf41eaba09 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:57.038942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7dd0587b-0804-4fbd-91bd-dacf41eaba09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:57.069796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0587b-0804-4fbd-91bd-dacf41eaba09 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:57.098894 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:57.101494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:57.102165 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:57.102847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] PATH_INFO: `/v3/projects/b3c33dc11d9d41eabd986b4a4c536616` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:57.108646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c33dc11d9d41eabd986b4a4c536616)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:57.112802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:57.140410 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3621e87-2c63-46d9-9bd0-5cb206a7aeb3 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:57.144057 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f3621e87-2c63-46d9-9bd0-5cb206a7aeb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:57.157299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:57.177786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3621e87-2c63-46d9-9bd0-5cb206a7aeb3 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:57.208185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:57.209164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:57.209738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:57.210163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] PATH_INFO: `/v3/projects/54ac2f0a43324c2eb43ba3e70e7a9b4a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:57.215917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54ac2f0a43324c2eb43ba3e70e7a9b4a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:57.219820 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:57.285806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c33dc11d9d41eabd986b4a4c536616', 'request_id': 'req-88b0b56a-19c8-4730-a395-40051673b42c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:57.300857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c33dc11d9d41eabd986b4a4c536616', 'request_id': 'req-88b0b56a-19c8-4730-a395-40051673b42c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:57.311782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c33dc11d9d41eabd986b4a4c536616', 'request_id': 'req-88b0b56a-19c8-4730-a395-40051673b42c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:57.341211 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] The token cache is being invalidate because project b3c33dc11d9d41eabd986b4a4c536616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:57.341559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:57.345386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:57.365104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1077/2189] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:57 2022] DELETE /identity/v3/projects/b3c33dc11d9d41eabd986b4a4c536616 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:57.381454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54ac2f0a43324c2eb43ba3e70e7a9b4a', 'request_id': 'req-8b5050e3-948f-4f48-a3cb-b43ed634d405'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:57.406236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54ac2f0a43324c2eb43ba3e70e7a9b4a', 'request_id': 'req-8b5050e3-948f-4f48-a3cb-b43ed634d405'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:57.418937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54ac2f0a43324c2eb43ba3e70e7a9b4a', 'request_id': 'req-8b5050e3-948f-4f48-a3cb-b43ed634d405'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:57.422268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:57.445094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] The token cache is being invalidate because project 54ac2f0a43324c2eb43ba3e70e7a9b4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:57.447596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:57.449774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:57.456554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1113/2190] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:57 2022] DELETE /identity/v3/projects/54ac2f0a43324c2eb43ba3e70e7a9b4a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:57.460935 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:57.462995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:57.657679 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0b56a-19c8-4730-a395-40051673b42c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:57.799761 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1d57c-3b61-47fc-852e-09ec123f5d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:57.800690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d57c-3b61-47fc-852e-09ec123f5d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:57.800986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d57c-3b61-47fc-852e-09ec123f5d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:57.801265 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1d57c-3b61-47fc-852e-09ec123f5d14 admin admin] PATH_INFO: `/v3/users/2422d9e1db3a44fea94d2de4d6b81d48` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:57.804419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1d57c-3b61-47fc-852e-09ec123f5d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2422d9e1db3a44fea94d2de4d6b81d48)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:57.807032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1d57c-3b61-47fc-852e-09ec123f5d14 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:57.947492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a0f1d57c-3b61-47fc-852e-09ec123f5d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2422d9e1db3a44fea94d2de4d6b81d48'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:57.967434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:57.989164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a0f1d57c-3b61-47fc-852e-09ec123f5d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2422d9e1db3a44fea94d2de4d6b81d48'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:58.009663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a0f1d57c-3b61-47fc-852e-09ec123f5d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2422d9e1db3a44fea94d2de4d6b81d48'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:58.034487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a0f1d57c-3b61-47fc-852e-09ec123f5d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2422d9e1db3a44fea94d2de4d6b81d48'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:58.039073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a0f1d57c-3b61-47fc-852e-09ec123f5d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2422d9e1db3a44fea94d2de4d6b81d48'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:58.053226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1078/2191] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:38:57 2022] DELETE /identity/v3/users/2422d9e1db3a44fea94d2de4d6b81d48 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:59.057405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:59.059613 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:59.091747 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5050e3-948f-4f48-a3cb-b43ed634d405 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:59.242901 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:59.244401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:59.245117 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:59.245807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82 admin admin] PATH_INFO: `/v3/projects/2a2da1a86b5a4751b2acf6791ece0167` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:59.249816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a2da1a86b5a4751b2acf6791ece0167)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:59.264338 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:59.401554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a2da1a86b5a4751b2acf6791ece0167', 'request_id': 'req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:59.410074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a2da1a86b5a4751b2acf6791ece0167', 'request_id': 'req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:59.423064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a2da1a86b5a4751b2acf6791ece0167', 'request_id': 'req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:59.458449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82 admin admin] The token cache is being invalidate because project 2a2da1a86b5a4751b2acf6791ece0167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:38:59.459076 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:59.459782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-77d56aa3-85dd-42d8-94a2-0d655cbe2b82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:38:59.461773 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1114/2192] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:38:59 2022] DELETE /identity/v3/projects/2a2da1a86b5a4751b2acf6791ece0167 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:38:59.468461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5596dcc-138f-47c6-b920-04b2bbc7d00f None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:59.469072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5596dcc-138f-47c6-b920-04b2bbc7d00f None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:59.469072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5596dcc-138f-47c6-b920-04b2bbc7d00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:59.508489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e5596dcc-138f-47c6-b920-04b2bbc7d00f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:59.509808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e5596dcc-138f-47c6-b920-04b2bbc7d00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:59.606027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1079/2193] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:59.613778 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0b683-29cc-45b2-9684-ee4966ea3afb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:59.614620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0b683-29cc-45b2-9684-ee4966ea3afb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:59.615119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0b683-29cc-45b2-9684-ee4966ea3afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:59.637212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a6d0b683-29cc-45b2-9684-ee4966ea3afb None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:59.639265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a6d0b683-29cc-45b2-9684-ee4966ea3afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:59.653995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1115/2194] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:59.658667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5596dcc-138f-47c6-b920-04b2bbc7d00f None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:59.687661 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5596dcc-138f-47c6-b920-04b2bbc7d00f None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:59.712891 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f7370-c083-4f4b-890b-a0ff369ee180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:59.715697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f7370-c083-4f4b-890b-a0ff369ee180 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:59.717062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f7370-c083-4f4b-890b-a0ff369ee180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:59.717780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f7370-c083-4f4b-890b-a0ff369ee180 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:59.720802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f7370-c083-4f4b-890b-a0ff369ee180 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:38:59.721358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f7370-c083-4f4b-890b-a0ff369ee180 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:59.734783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f7370-c083-4f4b-890b-a0ff369ee180 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:59.773645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1080/2195] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:38:59.777762 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0b683-29cc-45b2-9684-ee4966ea3afb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:59.789102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0b683-29cc-45b2-9684-ee4966ea3afb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:59.795138 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb2493-e5f1-4b65-bb5c-0763950dda47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:59.796200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2493-e5f1-4b65-bb5c-0763950dda47 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:59.796690 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2493-e5f1-4b65-bb5c-0763950dda47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:59.797137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2493-e5f1-4b65-bb5c-0763950dda47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:59.798067 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb2493-e5f1-4b65-bb5c-0763950dda47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:59.800196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb2493-e5f1-4b65-bb5c-0763950dda47 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:59.875368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1116/2196] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:38:59 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:38:59.879744 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f7370-c083-4f4b-890b-a0ff369ee180 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:59.902936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f7370-c083-4f4b-890b-a0ff369ee180 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:00.078673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317240c-83aa-4837-b8cd-0c343ae453f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:00.079324 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317240c-83aa-4837-b8cd-0c343ae453f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:00.079757 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317240c-83aa-4837-b8cd-0c343ae453f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:00.080197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317240c-83aa-4837-b8cd-0c343ae453f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:00.080868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317240c-83aa-4837-b8cd-0c343ae453f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:00.083279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317240c-83aa-4837-b8cd-0c343ae453f3 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:00.113051 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1081/2197] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:38:59 2022] POST /identity/v3/users => generated 287 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:39:00.117255 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb2493-e5f1-4b65-bb5c-0763950dda47 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:00.133077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb2493-e5f1-4b65-bb5c-0763950dda47 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:00.145088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb63f01-1a67-4d23-aea0-a41c3ae43be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:00.146066 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb63f01-1a67-4d23-aea0-a41c3ae43be7 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:00.146517 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb63f01-1a67-4d23-aea0-a41c3ae43be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:00.146940 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb63f01-1a67-4d23-aea0-a41c3ae43be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:00.147765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb63f01-1a67-4d23-aea0-a41c3ae43be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:00.150107 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb63f01-1a67-4d23-aea0-a41c3ae43be7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:00.165478 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1117/2198] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:39:00.169162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317240c-83aa-4837-b8cd-0c343ae453f3 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:00.196002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317240c-83aa-4837-b8cd-0c343ae453f3 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:00.205828 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184bd18-8b33-470b-b73d-444a03ae97f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:00.208574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184bd18-8b33-470b-b73d-444a03ae97f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:00.208574 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184bd18-8b33-470b-b73d-444a03ae97f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:00.209084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184bd18-8b33-470b-b73d-444a03ae97f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:00.210627 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184bd18-8b33-470b-b73d-444a03ae97f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:00.213582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184bd18-8b33-470b-b73d-444a03ae97f6 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:00.241382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1082/2199] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:39:00.243898 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb63f01-1a67-4d23-aea0-a41c3ae43be7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:00.274585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb63f01-1a67-4d23-aea0-a41c3ae43be7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:00.289046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23fe6e-dfa7-4b39-901f-bd24bf81a654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:00.290101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23fe6e-dfa7-4b39-901f-bd24bf81a654 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:00.291335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23fe6e-dfa7-4b39-901f-bd24bf81a654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:00.291848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23fe6e-dfa7-4b39-901f-bd24bf81a654 admin admin] PATH_INFO: `/v3/projects/99a0cc2a19f0457a93b13f31c08e498b/users/df7f1514a2a34278a14e6399c594ed29/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:00.328017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23fe6e-dfa7-4b39-901f-bd24bf81a654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a0cc2a19f0457a93b13f31c08e498b, user_id=df7f1514a2a34278a14e6399c594ed29, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:00.331221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23fe6e-dfa7-4b39-901f-bd24bf81a654 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:00.351491 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1118/2200] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:39:00 2022] PUT /identity/v3/projects/99a0cc2a19f0457a93b13f31c08e498b/users/df7f1514a2a34278a14e6399c594ed29/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:00.366072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2a44c-fc2c-4b7f-bd66-2f2dba766baa None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:00.366904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2a44c-fc2c-4b7f-bd66-2f2dba766baa None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:00.367581 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2a44c-fc2c-4b7f-bd66-2f2dba766baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:00.406467 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-3ff2a44c-fc2c-4b7f-bd66-2f2dba766baa None None] MFA Rules not processed for user `df7f1514a2a34278a14e6399c594ed29`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:00.407875 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3ff2a44c-fc2c-4b7f-bd66-2f2dba766baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:00.448694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:00.554518 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1083/2201] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:39:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:00.571705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23fe6e-dfa7-4b39-901f-bd24bf81a654 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:00.574995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6e23fe6e-dfa7-4b39-901f-bd24bf81a654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:00.609407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:00.672384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23fe6e-dfa7-4b39-901f-bd24bf81a654 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:00.679560 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7371da1-7879-48e3-ae59-5fe150c6bd3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:00.680686 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7371da1-7879-48e3-ae59-5fe150c6bd3c service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:00.681247 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7371da1-7879-48e3-ae59-5fe150c6bd3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:00.681762 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7371da1-7879-48e3-ae59-5fe150c6bd3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:00.711622 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7371da1-7879-48e3-ae59-5fe150c6bd3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:00.714417 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7371da1-7879-48e3-ae59-5fe150c6bd3c service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:00.731398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1119/2202] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:00.756772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2a44c-fc2c-4b7f-bd66-2f2dba766baa None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:00.855060 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2a44c-fc2c-4b7f-bd66-2f2dba766baa None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:00.866789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e1fde-0a80-4c13-b5f8-68edce2f6784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:00.867473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e1fde-0a80-4c13-b5f8-68edce2f6784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:00.867748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e1fde-0a80-4c13-b5f8-68edce2f6784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:00.868052 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e1fde-0a80-4c13-b5f8-68edce2f6784 admin admin] PATH_INFO: `/v3/users/df7f1514a2a34278a14e6399c594ed29` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:00.869251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e1fde-0a80-4c13-b5f8-68edce2f6784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df7f1514a2a34278a14e6399c594ed29)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:00.871381 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e1fde-0a80-4c13-b5f8-68edce2f6784 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:00.950411 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5b7e1fde-0a80-4c13-b5f8-68edce2f6784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df7f1514a2a34278a14e6399c594ed29'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:00.979780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5b7e1fde-0a80-4c13-b5f8-68edce2f6784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df7f1514a2a34278a14e6399c594ed29'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:00.999238 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5b7e1fde-0a80-4c13-b5f8-68edce2f6784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df7f1514a2a34278a14e6399c594ed29'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:01.026273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5b7e1fde-0a80-4c13-b5f8-68edce2f6784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df7f1514a2a34278a14e6399c594ed29'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:01.037093 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5b7e1fde-0a80-4c13-b5f8-68edce2f6784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df7f1514a2a34278a14e6399c594ed29'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:01.050916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1084/2203] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:39:00 2022] DELETE /identity/v3/users/df7f1514a2a34278a14e6399c594ed29 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:01.053243 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f11e6-31bc-4d4c-9f5b-2ab3028f4586 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:01.054062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f11e6-31bc-4d4c-9f5b-2ab3028f4586 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:01.054533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f11e6-31bc-4d4c-9f5b-2ab3028f4586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:01.090071 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8a2f11e6-31bc-4d4c-9f5b-2ab3028f4586 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:01.091643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8a2f11e6-31bc-4d4c-9f5b-2ab3028f4586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:01.171714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1120/2204] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:01.181475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e1fde-0a80-4c13-b5f8-68edce2f6784 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:01.185721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5b7e1fde-0a80-4c13-b5f8-68edce2f6784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:01.258963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:01.300003 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e1fde-0a80-4c13-b5f8-68edce2f6784 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:01.314850 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06ba3e-b517-4533-99c0-c726a78ce72c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:01.315668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06ba3e-b517-4533-99c0-c726a78ce72c service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:01.317866 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06ba3e-b517-4533-99c0-c726a78ce72c service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:01.318244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06ba3e-b517-4533-99c0-c726a78ce72c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:01.347259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06ba3e-b517-4533-99c0-c726a78ce72c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:01.351193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06ba3e-b517-4533-99c0-c726a78ce72c service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:01.377280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1085/2205] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:39:01.880822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f11e6-31bc-4d4c-9f5b-2ab3028f4586 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:01.882397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8a2f11e6-31bc-4d4c-9f5b-2ab3028f4586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:01.909546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f11e6-31bc-4d4c-9f5b-2ab3028f4586 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:01.925933 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503287b-a758-47f6-8314-425cb3ff5403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:01.926740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503287b-a758-47f6-8314-425cb3ff5403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:01.927110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503287b-a758-47f6-8314-425cb3ff5403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:01.927468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503287b-a758-47f6-8314-425cb3ff5403 admin admin] PATH_INFO: `/v3/projects/99a0cc2a19f0457a93b13f31c08e498b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:01.929032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503287b-a758-47f6-8314-425cb3ff5403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99a0cc2a19f0457a93b13f31c08e498b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:01.931106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503287b-a758-47f6-8314-425cb3ff5403 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:01.992007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e503287b-a758-47f6-8314-425cb3ff5403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99a0cc2a19f0457a93b13f31c08e498b', 'request_id': 'req-e503287b-a758-47f6-8314-425cb3ff5403'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:02.001175 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e503287b-a758-47f6-8314-425cb3ff5403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99a0cc2a19f0457a93b13f31c08e498b', 'request_id': 'req-e503287b-a758-47f6-8314-425cb3ff5403'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:02.014891 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e503287b-a758-47f6-8314-425cb3ff5403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99a0cc2a19f0457a93b13f31c08e498b', 'request_id': 'req-e503287b-a758-47f6-8314-425cb3ff5403'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:02.031945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e503287b-a758-47f6-8314-425cb3ff5403 admin admin] The token cache is being invalidate because project 99a0cc2a19f0457a93b13f31c08e498b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:39:02.032527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e503287b-a758-47f6-8314-425cb3ff5403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:02.033373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e503287b-a758-47f6-8314-425cb3ff5403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:39:02.039276 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1121/2206] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:39:01 2022] DELETE /identity/v3/projects/99a0cc2a19f0457a93b13f31c08e498b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:39:02.041690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba0b4f-a80b-43e8-a9a8-6c55f7d81ad3 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:02.042186 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba0b4f-a80b-43e8-a9a8-6c55f7d81ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:02.042333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba0b4f-a80b-43e8-a9a8-6c55f7d81ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:02.067294 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-09ba0b4f-a80b-43e8-a9a8-6c55f7d81ad3 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:02.067294 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-09ba0b4f-a80b-43e8-a9a8-6c55f7d81ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:02.191945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1086/2207] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:39:02.194502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58674e82-8ce0-4969-bbaa-f1e8f2e160d9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:02.194977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58674e82-8ce0-4969-bbaa-f1e8f2e160d9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:02.195379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58674e82-8ce0-4969-bbaa-f1e8f2e160d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:02.231693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-58674e82-8ce0-4969-bbaa-f1e8f2e160d9 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:02.239022 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-58674e82-8ce0-4969-bbaa-f1e8f2e160d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:02.248728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1122/2208] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:02.252867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba0b4f-a80b-43e8-a9a8-6c55f7d81ad3 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:02.264375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba0b4f-a80b-43e8-a9a8-6c55f7d81ad3 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:02.271545 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d5546-dad8-49cb-b605-f3de511f4e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:02.272922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d5546-dad8-49cb-b605-f3de511f4e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:02.273436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d5546-dad8-49cb-b605-f3de511f4e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:02.273871 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d5546-dad8-49cb-b605-f3de511f4e43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:02.274688 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d5546-dad8-49cb-b605-f3de511f4e43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:39:02.275131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d5546-dad8-49cb-b605-f3de511f4e43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:02.276974 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d5546-dad8-49cb-b605-f3de511f4e43 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:02.292987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1087/2209] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:39:02.297835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58674e82-8ce0-4969-bbaa-f1e8f2e160d9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:02.323463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58674e82-8ce0-4969-bbaa-f1e8f2e160d9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:02.339704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9268085-ef04-4655-a00e-11379d5ebbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:02.341196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9268085-ef04-4655-a00e-11379d5ebbd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:02.341483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9268085-ef04-4655-a00e-11379d5ebbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:02.341925 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9268085-ef04-4655-a00e-11379d5ebbd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:02.342569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9268085-ef04-4655-a00e-11379d5ebbd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:02.346172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9268085-ef04-4655-a00e-11379d5ebbd6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:02.397104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1123/2210] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:39:02 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:39:02.398865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d5546-dad8-49cb-b605-f3de511f4e43 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:02.415754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d5546-dad8-49cb-b605-f3de511f4e43 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:02.536232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cff603-eaf3-4058-b3a2-533ecef78f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:02.537405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cff603-eaf3-4058-b3a2-533ecef78f28 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:02.538267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cff603-eaf3-4058-b3a2-533ecef78f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:02.538461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cff603-eaf3-4058-b3a2-533ecef78f28 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:02.539258 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cff603-eaf3-4058-b3a2-533ecef78f28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:02.541760 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cff603-eaf3-4058-b3a2-533ecef78f28 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:02.585280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1088/2211] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:39:02 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:39:02.587245 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9268085-ef04-4655-a00e-11379d5ebbd6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:02.604591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9268085-ef04-4655-a00e-11379d5ebbd6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:02.611090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc07a1-f3f0-498c-a54b-b1257bffd48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:02.612121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc07a1-f3f0-498c-a54b-b1257bffd48c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:02.612548 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc07a1-f3f0-498c-a54b-b1257bffd48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:02.612950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc07a1-f3f0-498c-a54b-b1257bffd48c admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:02.613751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc07a1-f3f0-498c-a54b-b1257bffd48c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:02.615572 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc07a1-f3f0-498c-a54b-b1257bffd48c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:02.629165 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:02.644082 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1124/2212] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:39:02.646512 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cff603-eaf3-4058-b3a2-533ecef78f28 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:02.661911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cff603-eaf3-4058-b3a2-533ecef78f28 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:02.671226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde0c26-b61b-426b-b6a7-1327f5cf7dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:02.671908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde0c26-b61b-426b-b6a7-1327f5cf7dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:02.672183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde0c26-b61b-426b-b6a7-1327f5cf7dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:02.672455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde0c26-b61b-426b-b6a7-1327f5cf7dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:02.673134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde0c26-b61b-426b-b6a7-1327f5cf7dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:02.681960 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde0c26-b61b-426b-b6a7-1327f5cf7dd4 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:02.701255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1089/2213] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:39:02.706831 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc07a1-f3f0-498c-a54b-b1257bffd48c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:02.720943 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc07a1-f3f0-498c-a54b-b1257bffd48c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:02.728330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46990a47-9312-425b-bd78-ae26dcf5b685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:02.729050 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46990a47-9312-425b-bd78-ae26dcf5b685 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:02.729327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46990a47-9312-425b-bd78-ae26dcf5b685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:02.729596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46990a47-9312-425b-bd78-ae26dcf5b685 admin admin] PATH_INFO: `/v3/projects/d23aee15763f4997a71fa7e6864aa4a3/users/94fa5cd826e24f3992b537e56dc4e1da/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:02.797654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46990a47-9312-425b-bd78-ae26dcf5b685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23aee15763f4997a71fa7e6864aa4a3, user_id=94fa5cd826e24f3992b537e56dc4e1da, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:02.800633 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46990a47-9312-425b-bd78-ae26dcf5b685 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:02.867245 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1125/2214] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:39:02 2022] PUT /identity/v3/projects/d23aee15763f4997a71fa7e6864aa4a3/users/94fa5cd826e24f3992b537e56dc4e1da/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:39:02.873220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86057bad-3be8-4519-b5a6-ff3c95856bce None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:02.873220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86057bad-3be8-4519-b5a6-ff3c95856bce None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:02.873220 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86057bad-3be8-4519-b5a6-ff3c95856bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:02.897157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-86057bad-3be8-4519-b5a6-ff3c95856bce None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:02.907641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-86057bad-3be8-4519-b5a6-ff3c95856bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:03.018336 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:03.039065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1090/2215] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:03.046733 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46990a47-9312-425b-bd78-ae26dcf5b685 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:03.050213 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-46990a47-9312-425b-bd78-ae26dcf5b685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:03.146238 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46990a47-9312-425b-bd78-ae26dcf5b685 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:03.161611 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e959f-daaf-413d-8c9f-6b7bce33107e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:03.162263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e959f-daaf-413d-8c9f-6b7bce33107e service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:03.162729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e959f-daaf-413d-8c9f-6b7bce33107e service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:03.162999 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e959f-daaf-413d-8c9f-6b7bce33107e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:03.185154 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e959f-daaf-413d-8c9f-6b7bce33107e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:03.187026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e959f-daaf-413d-8c9f-6b7bce33107e service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:03.203653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1126/2216] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:04.255194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a86e84-4e3b-4eab-b91a-d8092574abba None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:04.255765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a86e84-4e3b-4eab-b91a-d8092574abba None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:04.255848 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a86e84-4e3b-4eab-b91a-d8092574abba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:04.289605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-65a86e84-4e3b-4eab-b91a-d8092574abba None None] MFA Rules not processed for user `94fa5cd826e24f3992b537e56dc4e1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:04.290867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-65a86e84-4e3b-4eab-b91a-d8092574abba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:04.423035 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1091/2217] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:39:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:04.438214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e959f-daaf-413d-8c9f-6b7bce33107e service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:04.440405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-310e959f-daaf-413d-8c9f-6b7bce33107e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:04.520422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e959f-daaf-413d-8c9f-6b7bce33107e service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:04.525073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb1093-a1fb-4f5a-948c-ebdb67b71eaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:04.525873 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb1093-a1fb-4f5a-948c-ebdb67b71eaa service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:04.526243 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb1093-a1fb-4f5a-948c-ebdb67b71eaa service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:04.526587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb1093-a1fb-4f5a-948c-ebdb67b71eaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:04.536973 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb1093-a1fb-4f5a-948c-ebdb67b71eaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:04.538522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb1093-a1fb-4f5a-948c-ebdb67b71eaa service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:04.549469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1127/2218] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:06.013391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fa349-f3b8-4aa8-9711-caa51f06a1cd None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:06.014068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fa349-f3b8-4aa8-9711-caa51f06a1cd None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:06.014068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fa349-f3b8-4aa8-9711-caa51f06a1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:06.036975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-231fa349-f3b8-4aa8-9711-caa51f06a1cd None None] MFA Rules not processed for user `94fa5cd826e24f3992b537e56dc4e1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:06.038234 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-231fa349-f3b8-4aa8-9711-caa51f06a1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:06.046171 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1092/2219] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:39:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:10.845260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42aa946-84b1-478f-9655-d88df46fc440 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:10.845807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42aa946-84b1-478f-9655-d88df46fc440 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:10.845807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42aa946-84b1-478f-9655-d88df46fc440 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:10.847204 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1128/2220] 10.222.0.62 () {54 vars in 921 bytes} [Thu Sep 1 16:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:39:10.853264 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bc2f3-f74f-49d3-abe9-8c2cf9b97233 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:10.853834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bc2f3-f74f-49d3-abe9-8c2cf9b97233 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:10.853834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bc2f3-f74f-49d3-abe9-8c2cf9b97233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:10.874557 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-067bc2f3-f74f-49d3-abe9-8c2cf9b97233 None None] MFA Rules not processed for user `2912c02ff6db4cbc97c80cd923238136`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:10.875803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-067bc2f3-f74f-49d3-abe9-8c2cf9b97233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:10.993384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1093/2221] 10.222.0.62 () {58 vars in 1018 bytes} [Thu Sep 1 16:39:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:11.006010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42aa946-84b1-478f-9655-d88df46fc440 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:11.007974 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a42aa946-84b1-478f-9655-d88df46fc440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:11.083135 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42aa946-84b1-478f-9655-d88df46fc440 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:11.086806 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42780158-ff58-4b57-94a5-769d6f5d2978 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfe36a14a11493397e4018b773d3cc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:11.087559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42780158-ff58-4b57-94a5-769d6f5d2978 service glance] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:11.088039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42780158-ff58-4b57-94a5-769d6f5d2978 service glance] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:11.088478 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42780158-ff58-4b57-94a5-769d6f5d2978 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:11.103250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42780158-ff58-4b57-94a5-769d6f5d2978 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:11.104757 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42780158-ff58-4b57-94a5-769d6f5d2978 service glance] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:11.119324 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1129/2222] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:11.211115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6270ae-f048-407d-a927-9a14b39261a3 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:11.211353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6270ae-f048-407d-a927-9a14b39261a3 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:11.211648 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6270ae-f048-407d-a927-9a14b39261a3 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:11.213023 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1094/2223] 10.222.0.62 () {54 vars in 921 bytes} [Thu Sep 1 16:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:39:11.218994 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92003670-f77a-42b4-a01f-5db53d8599b5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:11.219228 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92003670-f77a-42b4-a01f-5db53d8599b5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:11.219597 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92003670-f77a-42b4-a01f-5db53d8599b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:11.254610 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-92003670-f77a-42b4-a01f-5db53d8599b5 None None] MFA Rules not processed for user `0718a95189434a5ea901c40bc868bea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:11.256108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-92003670-f77a-42b4-a01f-5db53d8599b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:11.265645 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1130/2224] 10.222.0.62 () {58 vars in 1018 bytes} [Thu Sep 1 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:11.280088 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6270ae-f048-407d-a927-9a14b39261a3 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:11.295072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6270ae-f048-407d-a927-9a14b39261a3 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:11.300227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed0188-d7bb-491f-9a8f-fc57a69eca25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:11.300964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed0188-d7bb-491f-9a8f-fc57a69eca25 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:11.301305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed0188-d7bb-491f-9a8f-fc57a69eca25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:11.301638 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed0188-d7bb-491f-9a8f-fc57a69eca25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:11.313512 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed0188-d7bb-491f-9a8f-fc57a69eca25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:11.315775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed0188-d7bb-491f-9a8f-fc57a69eca25 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:11.329749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1095/2225] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:11.430640 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f546a1-27e0-4794-8e0a-28150a6776a6 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:11.430919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f546a1-27e0-4794-8e0a-28150a6776a6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:11.431193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f546a1-27e0-4794-8e0a-28150a6776a6 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:11.432620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1131/2226] 10.222.0.62 () {54 vars in 921 bytes} [Thu Sep 1 16:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:39:11.439697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06871a9-ff63-469e-8cdd-3dd89b988de5 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:11.440939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06871a9-ff63-469e-8cdd-3dd89b988de5 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:11.440939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06871a9-ff63-469e-8cdd-3dd89b988de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:11.465140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-f06871a9-ff63-469e-8cdd-3dd89b988de5 None None] MFA Rules not processed for user `2912c02ff6db4cbc97c80cd923238136`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:11.466598 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f06871a9-ff63-469e-8cdd-3dd89b988de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:11.477974 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1096/2227] 10.222.0.62 () {58 vars in 1018 bytes} [Thu Sep 1 16:39:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:39:11.489978 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f546a1-27e0-4794-8e0a-28150a6776a6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:11.502734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f546a1-27e0-4794-8e0a-28150a6776a6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:11.506732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d551f-aab6-43a2-a088-3fa6550e62ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:11.507415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d551f-aab6-43a2-a088-3fa6550e62ce service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:11.507778 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d551f-aab6-43a2-a088-3fa6550e62ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:11.508155 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d551f-aab6-43a2-a088-3fa6550e62ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:11.520189 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d551f-aab6-43a2-a088-3fa6550e62ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:11.521640 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d551f-aab6-43a2-a088-3fa6550e62ce service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:11.534714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1132/2228] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:39:17.382854 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc93622-88d5-4649-bcf3-c2a9d6ed8d87 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:17.383368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc93622-88d5-4649-bcf3-c2a9d6ed8d87 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:17.383451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc93622-88d5-4649-bcf3-c2a9d6ed8d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:17.404909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-3bc93622-88d5-4649-bcf3-c2a9d6ed8d87 None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:17.406172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3bc93622-88d5-4649-bcf3-c2a9d6ed8d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:17.457779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:17.472477 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1097/2229] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:39:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:17.485804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d551f-aab6-43a2-a088-3fa6550e62ce service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:17.487366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-462d551f-aab6-43a2-a088-3fa6550e62ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:17.494384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:17.568681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d551f-aab6-43a2-a088-3fa6550e62ce service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:17.573789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558298ba-f869-4297-b2c7-618b09bf8e0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:17.574416 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558298ba-f869-4297-b2c7-618b09bf8e0a service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:17.575369 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558298ba-f869-4297-b2c7-618b09bf8e0a service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:17.575369 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558298ba-f869-4297-b2c7-618b09bf8e0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:17.585401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558298ba-f869-4297-b2c7-618b09bf8e0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:17.587008 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558298ba-f869-4297-b2c7-618b09bf8e0a service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:17.596326 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1133/2230] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:39:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:39:27.464237 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:27.593216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=30 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:33.000241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc93622-88d5-4649-bcf3-c2a9d6ed8d87 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:33.001872 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3bc93622-88d5-4649-bcf3-c2a9d6ed8d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:33.095366 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc93622-88d5-4649-bcf3-c2a9d6ed8d87 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:33.100657 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbc3f8-8cc8-4061-a87c-c5399738190d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6281dcf2927a4ed2870125df8801be78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:33.101309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbc3f8-8cc8-4061-a87c-c5399738190d service placement] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:33.101575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbc3f8-8cc8-4061-a87c-c5399738190d service placement] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:33.101842 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbc3f8-8cc8-4061-a87c-c5399738190d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:33.103572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1dbbc3f8-8cc8-4061-a87c-c5399738190d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:33.116907 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbc3f8-8cc8-4061-a87c-c5399738190d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:33.118623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbc3f8-8cc8-4061-a87c-c5399738190d service placement] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:33.130107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1098/2231] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:38.587634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e2219-10b7-42ff-975e-4fba7696e9c1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:38.588383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e2219-10b7-42ff-975e-4fba7696e9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:38.588383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e2219-10b7-42ff-975e-4fba7696e9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:38.609210 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d98e2219-10b7-42ff-975e-4fba7696e9c1 None None] MFA Rules not processed for user `94fa5cd826e24f3992b537e56dc4e1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:38.610450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d98e2219-10b7-42ff-975e-4fba7696e9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:38.618239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1134/2232] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:39:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:43.125391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=32 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:44.795181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d655546-73e0-47c3-bc4c-5ffd063e5ae1 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:44.795772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d655546-73e0-47c3-bc4c-5ffd063e5ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:44.795772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d655546-73e0-47c3-bc4c-5ffd063e5ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:44.821482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6d655546-73e0-47c3-bc4c-5ffd063e5ae1 None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:44.823244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6d655546-73e0-47c3-bc4c-5ffd063e5ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:44.833262 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1099/2233] 10.222.0.62 () {58 vars in 967 bytes} [Thu Sep 1 16:39:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:44.847252 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e2219-10b7-42ff-975e-4fba7696e9c1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:44.859910 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e2219-10b7-42ff-975e-4fba7696e9c1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:44.865309 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6cca2-2130-437f-b428-622ce11bb7b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:44.865901 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6cca2-2130-437f-b428-622ce11bb7b5 service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:44.866171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6cca2-2130-437f-b428-622ce11bb7b5 service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:44.866438 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6cca2-2130-437f-b428-622ce11bb7b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:44.875279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6cca2-2130-437f-b428-622ce11bb7b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:44.877278 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6cca2-2130-437f-b428-622ce11bb7b5 service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:44.885763 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1135/2234] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:39:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:39:54.811867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:54.882310 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=9 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:02.006918 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedc1c1-aec7-41c9-9b39-db240d275666 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:02.007561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedc1c1-aec7-41c9-9b39-db240d275666 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:02.007561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedc1c1-aec7-41c9-9b39-db240d275666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:02.034224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-8bedc1c1-aec7-41c9-9b39-db240d275666 None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:02.035428 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8bedc1c1-aec7-41c9-9b39-db240d275666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:02.043366 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1100/2235] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:40:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:02.074125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6cca2-2130-437f-b428-622ce11bb7b5 service swift] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:02.085961 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6cca2-2130-437f-b428-622ce11bb7b5 service swift] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:02.091658 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbaa9d-541f-42fe-a7bc-425aa5068ebe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:02.092525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbaa9d-541f-42fe-a7bc-425aa5068ebe service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:02.092810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbaa9d-541f-42fe-a7bc-425aa5068ebe service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:02.093151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbaa9d-541f-42fe-a7bc-425aa5068ebe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:02.102938 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbaa9d-541f-42fe-a7bc-425aa5068ebe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:02.104962 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbaa9d-541f-42fe-a7bc-425aa5068ebe service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:02.116297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1136/2236] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:40:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:40:12.025740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:12.119424 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:18.680287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2987f83-05f7-48a0-a11e-40083c8bcb92 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:18.681449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2987f83-05f7-48a0-a11e-40083c8bcb92 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:18.681978 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2987f83-05f7-48a0-a11e-40083c8bcb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:18.706258 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-c2987f83-05f7-48a0-a11e-40083c8bcb92 None None] MFA Rules not processed for user `3a9b222396b1438f9fa0af0e197a9293`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:18.708218 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c2987f83-05f7-48a0-a11e-40083c8bcb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:18.777241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1101/2237] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:40:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:20.805104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae17196-71b6-48e3-baff-4c08abe4e040 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:20.805648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae17196-71b6-48e3-baff-4c08abe4e040 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:20.805739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae17196-71b6-48e3-baff-4c08abe4e040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:20.826066 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5ae17196-71b6-48e3-baff-4c08abe4e040 None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:20.827814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5ae17196-71b6-48e3-baff-4c08abe4e040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:20.836301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1137/2238] 10.222.0.62 () {58 vars in 967 bytes} [Thu Sep 1 16:40:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:20.850799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2987f83-05f7-48a0-a11e-40083c8bcb92 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:20.863668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2987f83-05f7-48a0-a11e-40083c8bcb92 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:20.872461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68eb0a7-d04a-4a7e-81dd-b6325212b0e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:20.873307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68eb0a7-d04a-4a7e-81dd-b6325212b0e6 service swift] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:20.875099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68eb0a7-d04a-4a7e-81dd-b6325212b0e6 service swift] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:20.875425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68eb0a7-d04a-4a7e-81dd-b6325212b0e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:20.888858 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68eb0a7-d04a-4a7e-81dd-b6325212b0e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:20.891168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68eb0a7-d04a-4a7e-81dd-b6325212b0e6 service swift] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:20.902025 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1102/2239] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:40:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:40:22.960272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c8154-61a7-4d24-ac70-60573008374b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:22.961283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c8154-61a7-4d24-ac70-60573008374b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:22.961623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c8154-61a7-4d24-ac70-60573008374b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:22.963161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1138/2240] 10.222.0.62 () {54 vars in 921 bytes} [Thu Sep 1 16:40:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:40:22.995870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0b7e2-1e43-444b-b87e-f106572de9eb None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:22.996703 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0b7e2-1e43-444b-b87e-f106572de9eb None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:22.996703 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0b7e2-1e43-444b-b87e-f106572de9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:23.023710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-64a0b7e2-1e43-444b-b87e-f106572de9eb None None] MFA Rules not processed for user `6281dcf2927a4ed2870125df8801be78`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:23.025153 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-64a0b7e2-1e43-444b-b87e-f106572de9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:23.041793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c8154-61a7-4d24-ac70-60573008374b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:23.046512 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1103/2241] 10.222.0.62 () {58 vars in 1018 bytes} [Thu Sep 1 16:40:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:23.061211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c8154-61a7-4d24-ac70-60573008374b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:23.064230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0b7e2-1e43-444b-b87e-f106572de9eb None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:23.068328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7e5e1-9f70-474e-84d4-1953e0178d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:23.070294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7e5e1-9f70-474e-84d4-1953e0178d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:23.070605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7e5e1-9f70-474e-84d4-1953e0178d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:23.071149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7e5e1-9f70-474e-84d4-1953e0178d65 admin admin] PATH_INFO: `/v3/users/94fa5cd826e24f3992b537e56dc4e1da` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:23.072819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7e5e1-9f70-474e-84d4-1953e0178d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94fa5cd826e24f3992b537e56dc4e1da)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:23.075411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7e5e1-9f70-474e-84d4-1953e0178d65 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:23.077976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0b7e2-1e43-444b-b87e-f106572de9eb None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:23.088363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2198f-8213-44d0-b7f8-e82556bff949 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6281dcf2927a4ed2870125df8801be78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:23.089107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2198f-8213-44d0-b7f8-e82556bff949 service placement] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:23.090838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2198f-8213-44d0-b7f8-e82556bff949 service placement] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:23.091094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2198f-8213-44d0-b7f8-e82556bff949 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:23.102555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2198f-8213-44d0-b7f8-e82556bff949 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:23.104446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2198f-8213-44d0-b7f8-e82556bff949 service placement] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:23.116995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1104/2242] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:40:23.136421 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ccc7e5e1-9f70-474e-84d4-1953e0178d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94fa5cd826e24f3992b537e56dc4e1da'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.146929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ccc7e5e1-9f70-474e-84d4-1953e0178d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94fa5cd826e24f3992b537e56dc4e1da'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.147603 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ccc7e5e1-9f70-474e-84d4-1953e0178d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94fa5cd826e24f3992b537e56dc4e1da'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.168727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ccc7e5e1-9f70-474e-84d4-1953e0178d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94fa5cd826e24f3992b537e56dc4e1da'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.177726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ccc7e5e1-9f70-474e-84d4-1953e0178d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94fa5cd826e24f3992b537e56dc4e1da'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.184863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1139/2243] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:40:23 2022] DELETE /identity/v3/users/94fa5cd826e24f3992b537e56dc4e1da => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:23.369495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2198f-8213-44d0-b7f8-e82556bff949 service placement] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:23.384986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2198f-8213-44d0-b7f8-e82556bff949 service placement] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:23.444337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:23.445054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:23.445288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:23.445564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77 admin admin] PATH_INFO: `/v3/projects/d23aee15763f4997a71fa7e6864aa4a3` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:23.446710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d23aee15763f4997a71fa7e6864aa4a3)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:23.448698 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:23.506879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd23aee15763f4997a71fa7e6864aa4a3', 'request_id': 'req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.518559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd23aee15763f4997a71fa7e6864aa4a3', 'request_id': 'req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.525327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd23aee15763f4997a71fa7e6864aa4a3', 'request_id': 'req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.537779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77 admin admin] The token cache is being invalidate because project d23aee15763f4997a71fa7e6864aa4a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:40:23.538070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.538800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-aaa6c137-8aba-4c83-baf1-49c9ca7a9a77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:23.540596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1105/2244] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:40:23 2022] DELETE /identity/v3/projects/d23aee15763f4997a71fa7e6864aa4a3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:28.773494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c85b-3db7-41f5-b976-3462b0108215 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:28.774084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c85b-3db7-41f5-b976-3462b0108215 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:28.774084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c85b-3db7-41f5-b976-3462b0108215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:28.795929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5e08c85b-3db7-41f5-b976-3462b0108215 None None] MFA Rules not processed for user `52760b6736e14ccd8df8aca0015cb292`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:28.797676 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5e08c85b-3db7-41f5-b976-3462b0108215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:28.858908 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1140/2245] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:40:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:33.534924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:38.851504 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:40.884618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556dc97-e6bc-4376-a020-74b43a8c415b None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:40.885127 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556dc97-e6bc-4376-a020-74b43a8c415b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:40.885251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556dc97-e6bc-4376-a020-74b43a8c415b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:40.905745 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-5556dc97-e6bc-4376-a020-74b43a8c415b None None] MFA Rules not processed for user `849f875ede78424c81919df6d9c152dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:40.906970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5556dc97-e6bc-4376-a020-74b43a8c415b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:40.979499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1106/2246] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:40:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:50.972066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=25 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:02.454362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c9521-f396-4525-9b33-7b206d363bba None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:02.454928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c9521-f396-4525-9b33-7b206d363bba None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:02.454928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c9521-f396-4525-9b33-7b206d363bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:02.475019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-502c9521-f396-4525-9b33-7b206d363bba None None] MFA Rules not processed for user `2c1c3f8f28b147fc838533baf62b72af`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:02.476255 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-502c9521-f396-4525-9b33-7b206d363bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:02.533524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1141/2247] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:09.630342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556dc97-e6bc-4376-a020-74b43a8c415b None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:09.631786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5556dc97-e6bc-4376-a020-74b43a8c415b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:09.696535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556dc97-e6bc-4376-a020-74b43a8c415b None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:09.704242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d07cb5-faee-41ba-830f-f82d1d6ea225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:09.704979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d07cb5-faee-41ba-830f-f82d1d6ea225 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:09.705297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d07cb5-faee-41ba-830f-f82d1d6ea225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:09.705730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d07cb5-faee-41ba-830f-f82d1d6ea225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:09.707763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f0d07cb5-faee-41ba-830f-f82d1d6ea225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:09.791193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d07cb5-faee-41ba-830f-f82d1d6ea225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:09.793352 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d07cb5-faee-41ba-830f-f82d1d6ea225 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:09.806322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1107/2248] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:10.713327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c9521-f396-4525-9b33-7b206d363bba None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:10.729543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c9521-f396-4525-9b33-7b206d363bba None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:10.735645 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd225a-ef04-4e97-8bde-3392a1db7b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:10.736280 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd225a-ef04-4e97-8bde-3392a1db7b56 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:10.736550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd225a-ef04-4e97-8bde-3392a1db7b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:10.736808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd225a-ef04-4e97-8bde-3392a1db7b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:10.738819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f5fd225a-ef04-4e97-8bde-3392a1db7b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:10.759524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd225a-ef04-4e97-8bde-3392a1db7b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:10.760977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd225a-ef04-4e97-8bde-3392a1db7b56 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:10.773017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1142/2249] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:19.801495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=54 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:20.767341 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=31 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:24.619079 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d07cb5-faee-41ba-830f-f82d1d6ea225 service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:24.620633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f0d07cb5-faee-41ba-830f-f82d1d6ea225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:24.701001 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d07cb5-faee-41ba-830f-f82d1d6ea225 service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:24.705900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a311d37-c5c2-497a-9e9c-0ed239e36368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:24.706727 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a311d37-c5c2-497a-9e9c-0ed239e36368 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.707014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a311d37-c5c2-497a-9e9c-0ed239e36368 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.707295 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a311d37-c5c2-497a-9e9c-0ed239e36368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.709396 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9a311d37-c5c2-497a-9e9c-0ed239e36368 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:24.723154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a311d37-c5c2-497a-9e9c-0ed239e36368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:24.724632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a311d37-c5c2-497a-9e9c-0ed239e36368 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:24.735584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b314b17-54aa-463d-a136-7ce56466962d None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.736243 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1108/2250] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:24.736368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b314b17-54aa-463d-a136-7ce56466962d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.736368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b314b17-54aa-463d-a136-7ce56466962d None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.737536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1143/2251] 10.222.0.62 () {54 vars in 921 bytes} [Thu Sep 1 16:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:41:24.744542 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78134ad5-787f-4ce8-b476-28211e8aca38 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.744878 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78134ad5-787f-4ce8-b476-28211e8aca38 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.745158 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78134ad5-787f-4ce8-b476-28211e8aca38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.768900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-78134ad5-787f-4ce8-b476-28211e8aca38 None None] MFA Rules not processed for user `0718a95189434a5ea901c40bc868bea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:24.770274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-78134ad5-787f-4ce8-b476-28211e8aca38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:24.778163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1109/2252] 10.222.0.62 () {58 vars in 1018 bytes} [Thu Sep 1 16:41:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:24.791561 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b314b17-54aa-463d-a136-7ce56466962d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:24.794608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7b314b17-54aa-463d-a136-7ce56466962d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:24.814693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b314b17-54aa-463d-a136-7ce56466962d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:24.819333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44423897-5e5a-45af-92f7-fec85a2fdfdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:24.820028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44423897-5e5a-45af-92f7-fec85a2fdfdf service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.820306 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44423897-5e5a-45af-92f7-fec85a2fdfdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.820624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44423897-5e5a-45af-92f7-fec85a2fdfdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.828630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44423897-5e5a-45af-92f7-fec85a2fdfdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:24.830089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44423897-5e5a-45af-92f7-fec85a2fdfdf service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:24.838322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1144/2253] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:24.932953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e29155-ba76-4d28-a6cb-f249d69147c5 None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.933222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e29155-ba76-4d28-a6cb-f249d69147c5 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.933511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e29155-ba76-4d28-a6cb-f249d69147c5 None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.934947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1110/2254] 10.222.0.62 () {54 vars in 921 bytes} [Thu Sep 1 16:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:41:24.941225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e1c4c-bf6f-487d-bd35-269d42398ebf None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.941647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e1c4c-bf6f-487d-bd35-269d42398ebf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.941938 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e1c4c-bf6f-487d-bd35-269d42398ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.965506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7b3e1c4c-bf6f-487d-bd35-269d42398ebf None None] MFA Rules not processed for user `2912c02ff6db4cbc97c80cd923238136`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:24.966731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7b3e1c4c-bf6f-487d-bd35-269d42398ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:24.978408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1145/2255] 10.222.0.62 () {58 vars in 1018 bytes} [Thu Sep 1 16:41:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:24.989544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e29155-ba76-4d28-a6cb-f249d69147c5 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:25.001780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e29155-ba76-4d28-a6cb-f249d69147c5 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:25.005891 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13aaf0-dede-456c-b3cc-86594f9b0224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:25.006496 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13aaf0-dede-456c-b3cc-86594f9b0224 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:25.006770 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13aaf0-dede-456c-b3cc-86594f9b0224 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:25.007038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13aaf0-dede-456c-b3cc-86594f9b0224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:25.017675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13aaf0-dede-456c-b3cc-86594f9b0224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:25.019138 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13aaf0-dede-456c-b3cc-86594f9b0224 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:25.030147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1111/2256] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:27.636971 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e1c4c-bf6f-487d-bd35-269d42398ebf None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:27.638509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7b3e1c4c-bf6f-487d-bd35-269d42398ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:27.651912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e1c4c-bf6f-487d-bd35-269d42398ebf None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:27.656914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde0b01-1aaf-43f2-b226-87435db1fd75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:27.657570 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde0b01-1aaf-43f2-b226-87435db1fd75 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:27.657845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde0b01-1aaf-43f2-b226-87435db1fd75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:27.658119 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde0b01-1aaf-43f2-b226-87435db1fd75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:27.660431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bcde0b01-1aaf-43f2-b226-87435db1fd75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:27.730584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde0b01-1aaf-43f2-b226-87435db1fd75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:27.732054 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde0b01-1aaf-43f2-b226-87435db1fd75 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:27.741946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1146/2257] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:35.025337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=35 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:37.736992 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=37 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:38.159876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13aaf0-dede-456c-b3cc-86594f9b0224 service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:38.161436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-da13aaf0-dede-456c-b3cc-86594f9b0224 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:38.177090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13aaf0-dede-456c-b3cc-86594f9b0224 service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:38.183975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00904f-7718-48ca-b0d2-868afe8a04e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:38.184545 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00904f-7718-48ca-b0d2-868afe8a04e1 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:38.184818 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00904f-7718-48ca-b0d2-868afe8a04e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:38.185083 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00904f-7718-48ca-b0d2-868afe8a04e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:38.187262 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5c00904f-7718-48ca-b0d2-868afe8a04e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:38.198339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00904f-7718-48ca-b0d2-868afe8a04e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:38.200303 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00904f-7718-48ca-b0d2-868afe8a04e1 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:38.210533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1112/2258] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:39.786850 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde0b01-1aaf-43f2-b226-87435db1fd75 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:39.797277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde0b01-1aaf-43f2-b226-87435db1fd75 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:39.801369 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132e9eb-ab29-42dc-b62a-3d8fa99afd90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:39.801976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132e9eb-ab29-42dc-b62a-3d8fa99afd90 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.802247 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132e9eb-ab29-42dc-b62a-3d8fa99afd90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.802543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132e9eb-ab29-42dc-b62a-3d8fa99afd90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.805255 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2132e9eb-ab29-42dc-b62a-3d8fa99afd90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:39.885839 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132e9eb-ab29-42dc-b62a-3d8fa99afd90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:39.887323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132e9eb-ab29-42dc-b62a-3d8fa99afd90 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:39.903930 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1147/2259] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:39.911812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f4601-bb12-400b-80f0-d9d113730c62 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.912390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f4601-bb12-400b-80f0-d9d113730c62 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.912514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f4601-bb12-400b-80f0-d9d113730c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.932869 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-fd5f4601-bb12-400b-80f0-d9d113730c62 None None] MFA Rules not processed for user `3a9b222396b1438f9fa0af0e197a9293`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:39.934258 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-fd5f4601-bb12-400b-80f0-d9d113730c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:39.942987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1113/2260] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:40.335361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132e9eb-ab29-42dc-b62a-3d8fa99afd90 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:40.353009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132e9eb-ab29-42dc-b62a-3d8fa99afd90 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:40.359681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d751c03-e834-48a6-b575-595b428abf33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:40.360477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d751c03-e834-48a6-b575-595b428abf33 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:40.360910 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d751c03-e834-48a6-b575-595b428abf33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:40.361429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d751c03-e834-48a6-b575-595b428abf33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:40.363668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7d751c03-e834-48a6-b575-595b428abf33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:40.375254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d751c03-e834-48a6-b575-595b428abf33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:40.376921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d751c03-e834-48a6-b575-595b428abf33 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:40.398216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1148/2261] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:40.854168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f4601-bb12-400b-80f0-d9d113730c62 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:40.855864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-fd5f4601-bb12-400b-80f0-d9d113730c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:40.939810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f4601-bb12-400b-80f0-d9d113730c62 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:40.946595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93d767-10f4-4164-97d0-b4929df93204 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6281dcf2927a4ed2870125df8801be78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:40.947244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d767-10f4-4164-97d0-b4929df93204 service placement] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:40.947517 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d767-10f4-4164-97d0-b4929df93204 service placement] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:40.947782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d767-10f4-4164-97d0-b4929df93204 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:40.950046 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-cb93d767-10f4-4164-97d0-b4929df93204 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:40.962837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93d767-10f4-4164-97d0-b4929df93204 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:40.964815 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93d767-10f4-4164-97d0-b4929df93204 service placement] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:40.975317 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1114/2262] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:43.773422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d751c03-e834-48a6-b575-595b428abf33 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:43.774605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7d751c03-e834-48a6-b575-595b428abf33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:43.796893 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d751c03-e834-48a6-b575-595b428abf33 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:43.801495 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c748c-a907-4cc8-a7c1-86f7fbc90477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:43.802138 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c748c-a907-4cc8-a7c1-86f7fbc90477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:43.802412 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c748c-a907-4cc8-a7c1-86f7fbc90477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:43.802683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c748c-a907-4cc8-a7c1-86f7fbc90477 admin admin] PATH_INFO: `/v3/users/52760b6736e14ccd8df8aca0015cb292` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:43.806814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c748c-a907-4cc8-a7c1-86f7fbc90477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52760b6736e14ccd8df8aca0015cb292)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:43.806814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c748c-a907-4cc8-a7c1-86f7fbc90477 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:43.861615 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c83c748c-a907-4cc8-a7c1-86f7fbc90477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52760b6736e14ccd8df8aca0015cb292'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:43.870691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c83c748c-a907-4cc8-a7c1-86f7fbc90477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52760b6736e14ccd8df8aca0015cb292'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:43.871288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c83c748c-a907-4cc8-a7c1-86f7fbc90477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52760b6736e14ccd8df8aca0015cb292'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:43.884550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c83c748c-a907-4cc8-a7c1-86f7fbc90477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52760b6736e14ccd8df8aca0015cb292'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:43.891818 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c83c748c-a907-4cc8-a7c1-86f7fbc90477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52760b6736e14ccd8df8aca0015cb292'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:43.898086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1149/2263] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:43 2022] DELETE /identity/v3/users/52760b6736e14ccd8df8aca0015cb292 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:44.096495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93d767-10f4-4164-97d0-b4929df93204 service placement] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:44.110509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93d767-10f4-4164-97d0-b4929df93204 service placement] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:44.176207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48f691-a6bf-42e6-a700-02b6446c403b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:44.176847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48f691-a6bf-42e6-a700-02b6446c403b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:44.177118 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48f691-a6bf-42e6-a700-02b6446c403b admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:44.177383 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48f691-a6bf-42e6-a700-02b6446c403b admin admin] PATH_INFO: `/v3/projects/601c248b5d9b48488351cb4d9561a2f4` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:44.178622 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48f691-a6bf-42e6-a700-02b6446c403b admin admin] RBAC: Authorizing `identity:delete_project(project_id=601c248b5d9b48488351cb4d9561a2f4)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:44.180594 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48f691-a6bf-42e6-a700-02b6446c403b admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:44.275800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0b48f691-a6bf-42e6-a700-02b6446c403b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '601c248b5d9b48488351cb4d9561a2f4', 'request_id': 'req-0b48f691-a6bf-42e6-a700-02b6446c403b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:44.291656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0b48f691-a6bf-42e6-a700-02b6446c403b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '601c248b5d9b48488351cb4d9561a2f4', 'request_id': 'req-0b48f691-a6bf-42e6-a700-02b6446c403b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:44.301497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0b48f691-a6bf-42e6-a700-02b6446c403b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '601c248b5d9b48488351cb4d9561a2f4', 'request_id': 'req-0b48f691-a6bf-42e6-a700-02b6446c403b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:44.311356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:44.315579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0b48f691-a6bf-42e6-a700-02b6446c403b admin admin] The token cache is being invalidate because project 601c248b5d9b48488351cb4d9561a2f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:44.316056 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0b48f691-a6bf-42e6-a700-02b6446c403b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:44.317122 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0b48f691-a6bf-42e6-a700-02b6446c403b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:44.319020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1115/2264] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:44 2022] DELETE /identity/v3/projects/601c248b5d9b48488351cb4d9561a2f4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:44.326959 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74af2fa-ded1-4d4e-beb5-679d26949b84 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:44.327242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74af2fa-ded1-4d4e-beb5-679d26949b84 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:44.327684 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74af2fa-ded1-4d4e-beb5-679d26949b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:44.358378 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c74af2fa-ded1-4d4e-beb5-679d26949b84 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:44.360091 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c74af2fa-ded1-4d4e-beb5-679d26949b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:44.427388 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:44.449538 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1150/2265] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:44.456944 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4965ad1-07ab-4b4a-81f9-2de63caba3c5 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:44.457513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4965ad1-07ab-4b4a-81f9-2de63caba3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:44.457974 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4965ad1-07ab-4b4a-81f9-2de63caba3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:44.479006 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-c4965ad1-07ab-4b4a-81f9-2de63caba3c5 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:44.480622 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c4965ad1-07ab-4b4a-81f9-2de63caba3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:44.489996 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1116/2266] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:44.494540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74af2fa-ded1-4d4e-beb5-679d26949b84 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:44.507588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74af2fa-ded1-4d4e-beb5-679d26949b84 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:44.514132 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf231af-893e-4dd7-88bd-e8857a9b3687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:44.514862 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf231af-893e-4dd7-88bd-e8857a9b3687 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:44.515184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf231af-893e-4dd7-88bd-e8857a9b3687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:44.515541 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf231af-893e-4dd7-88bd-e8857a9b3687 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:44.516325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf231af-893e-4dd7-88bd-e8857a9b3687 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:41:44.516851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf231af-893e-4dd7-88bd-e8857a9b3687 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:44.518673 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf231af-893e-4dd7-88bd-e8857a9b3687 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:44.546706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1151/2267] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:41:44.551267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4965ad1-07ab-4b4a-81f9-2de63caba3c5 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:44.563576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4965ad1-07ab-4b4a-81f9-2de63caba3c5 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:44.570124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a124ab-f547-4f72-87c1-65fd2db1d312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:44.570922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a124ab-f547-4f72-87c1-65fd2db1d312 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:44.571272 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a124ab-f547-4f72-87c1-65fd2db1d312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:44.571611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a124ab-f547-4f72-87c1-65fd2db1d312 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:44.572383 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a124ab-f547-4f72-87c1-65fd2db1d312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:44.574936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a124ab-f547-4f72-87c1-65fd2db1d312 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:44.608579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1117/2268] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:44 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:44.612821 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf231af-893e-4dd7-88bd-e8857a9b3687 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:44.630827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf231af-893e-4dd7-88bd-e8857a9b3687 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:44.692000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cad1e-b455-47ad-9a20-8c81f8f2b2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:44.692648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cad1e-b455-47ad-9a20-8c81f8f2b2ae admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:44.692923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cad1e-b455-47ad-9a20-8c81f8f2b2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:44.693196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cad1e-b455-47ad-9a20-8c81f8f2b2ae admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:44.693875 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cad1e-b455-47ad-9a20-8c81f8f2b2ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:44.695797 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cad1e-b455-47ad-9a20-8c81f8f2b2ae admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:44.721595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1152/2269] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:44 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:44.725977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a124ab-f547-4f72-87c1-65fd2db1d312 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:44.736962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a124ab-f547-4f72-87c1-65fd2db1d312 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:44.742222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615fae6-d180-46dd-8d0e-0fb95ad24c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:44.742832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615fae6-d180-46dd-8d0e-0fb95ad24c11 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:44.743115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615fae6-d180-46dd-8d0e-0fb95ad24c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:44.743440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615fae6-d180-46dd-8d0e-0fb95ad24c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:44.744122 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615fae6-d180-46dd-8d0e-0fb95ad24c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:44.745744 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615fae6-d180-46dd-8d0e-0fb95ad24c11 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:44.760295 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1118/2270] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:41:44.761942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cad1e-b455-47ad-9a20-8c81f8f2b2ae admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:44.772300 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cad1e-b455-47ad-9a20-8c81f8f2b2ae admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:44.777215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5f8ca-b510-4dd5-b3a0-d8f4604c0f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:44.777888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5f8ca-b510-4dd5-b3a0-d8f4604c0f46 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:44.778229 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5f8ca-b510-4dd5-b3a0-d8f4604c0f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:44.778582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5f8ca-b510-4dd5-b3a0-d8f4604c0f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:44.779287 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5f8ca-b510-4dd5-b3a0-d8f4604c0f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:44.780986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5f8ca-b510-4dd5-b3a0-d8f4604c0f46 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:44.793010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1153/2271] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:44.797461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615fae6-d180-46dd-8d0e-0fb95ad24c11 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:44.808991 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615fae6-d180-46dd-8d0e-0fb95ad24c11 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:44.813831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21995b09-32f5-424c-9d26-cd7a38e7579a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:44.814492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21995b09-32f5-424c-9d26-cd7a38e7579a admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:44.814775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21995b09-32f5-424c-9d26-cd7a38e7579a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:44.815047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21995b09-32f5-424c-9d26-cd7a38e7579a admin admin] PATH_INFO: `/v3/projects/dc025fe8f2f944be83d8e52a94575238/users/0e46ac7f2c8343a0a81a9a9dca1b0d22/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:44.831067 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21995b09-32f5-424c-9d26-cd7a38e7579a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc025fe8f2f944be83d8e52a94575238, user_id=0e46ac7f2c8343a0a81a9a9dca1b0d22, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:44.834211 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21995b09-32f5-424c-9d26-cd7a38e7579a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:44.847967 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1119/2272] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:44 2022] PUT /identity/v3/projects/dc025fe8f2f944be83d8e52a94575238/users/0e46ac7f2c8343a0a81a9a9dca1b0d22/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:44.859330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c2fca-38af-4b0b-ab9a-550d27c7b966 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:44.859613 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c2fca-38af-4b0b-ab9a-550d27c7b966 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:44.859898 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c2fca-38af-4b0b-ab9a-550d27c7b966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:44.908451 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d79c2fca-38af-4b0b-ab9a-550d27c7b966 None None] MFA Rules not processed for user `0e46ac7f2c8343a0a81a9a9dca1b0d22`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:44.909950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d79c2fca-38af-4b0b-ab9a-550d27c7b966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:44.982101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1154/2273] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:44.993712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21995b09-32f5-424c-9d26-cd7a38e7579a admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:44.995530 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-21995b09-32f5-424c-9d26-cd7a38e7579a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:45.078332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21995b09-32f5-424c-9d26-cd7a38e7579a admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:45.085349 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcba8e1-cc92-47f2-884c-89064902ac00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:45.086084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcba8e1-cc92-47f2-884c-89064902ac00 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:45.086453 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcba8e1-cc92-47f2-884c-89064902ac00 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:45.086796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcba8e1-cc92-47f2-884c-89064902ac00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:45.095595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcba8e1-cc92-47f2-884c-89064902ac00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:45.097290 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcba8e1-cc92-47f2-884c-89064902ac00 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:45.107501 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1120/2274] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:45.335807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c2fca-38af-4b0b-ab9a-550d27c7b966 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:45.388743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:45.406041 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c2fca-38af-4b0b-ab9a-550d27c7b966 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:45.411366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f5a0a-15b2-4111-b3d1-c55b59b778b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:45.412043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f5a0a-15b2-4111-b3d1-c55b59b778b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:45.412354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f5a0a-15b2-4111-b3d1-c55b59b778b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:45.412649 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f5a0a-15b2-4111-b3d1-c55b59b778b8 admin admin] PATH_INFO: `/v3/users/0e46ac7f2c8343a0a81a9a9dca1b0d22` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:45.413939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f5a0a-15b2-4111-b3d1-c55b59b778b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e46ac7f2c8343a0a81a9a9dca1b0d22)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:45.415873 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f5a0a-15b2-4111-b3d1-c55b59b778b8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:45.469268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-412f5a0a-15b2-4111-b3d1-c55b59b778b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e46ac7f2c8343a0a81a9a9dca1b0d22'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:45.477144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-412f5a0a-15b2-4111-b3d1-c55b59b778b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e46ac7f2c8343a0a81a9a9dca1b0d22'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:45.477955 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-412f5a0a-15b2-4111-b3d1-c55b59b778b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e46ac7f2c8343a0a81a9a9dca1b0d22'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:45.489871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-412f5a0a-15b2-4111-b3d1-c55b59b778b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e46ac7f2c8343a0a81a9a9dca1b0d22'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:45.496891 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-412f5a0a-15b2-4111-b3d1-c55b59b778b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e46ac7f2c8343a0a81a9a9dca1b0d22'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:45.503216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1155/2275] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:45 2022] DELETE /identity/v3/users/0e46ac7f2c8343a0a81a9a9dca1b0d22 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:45.508217 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b36f6-3919-47fe-b82f-84d156344d1b None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:45.508559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b36f6-3919-47fe-b82f-84d156344d1b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:45.508896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b36f6-3919-47fe-b82f-84d156344d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:45.528995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-1c5b36f6-3919-47fe-b82f-84d156344d1b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:45.530396 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1c5b36f6-3919-47fe-b82f-84d156344d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:45.609876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1121/2276] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:45.619981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f5a0a-15b2-4111-b3d1-c55b59b778b8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:45.621700 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-412f5a0a-15b2-4111-b3d1-c55b59b778b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:45.683923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f5a0a-15b2-4111-b3d1-c55b59b778b8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:45.688157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849c120-f2ec-4a96-bb7f-b1861113fd7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:45.688775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849c120-f2ec-4a96-bb7f-b1861113fd7b service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:45.689047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849c120-f2ec-4a96-bb7f-b1861113fd7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:45.689314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849c120-f2ec-4a96-bb7f-b1861113fd7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:45.699454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849c120-f2ec-4a96-bb7f-b1861113fd7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:45.700911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849c120-f2ec-4a96-bb7f-b1861113fd7b service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:45.714150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1156/2277] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:46.011281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b36f6-3919-47fe-b82f-84d156344d1b None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:46.012815 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1c5b36f6-3919-47fe-b82f-84d156344d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:46.026877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b36f6-3919-47fe-b82f-84d156344d1b None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:46.032538 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd91403-b732-4add-935f-6f5a6569da1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:46.033222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd91403-b732-4add-935f-6f5a6569da1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:46.033530 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd91403-b732-4add-935f-6f5a6569da1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:46.033773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd91403-b732-4add-935f-6f5a6569da1a admin admin] PATH_INFO: `/v3/projects/dc025fe8f2f944be83d8e52a94575238` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:46.035539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd91403-b732-4add-935f-6f5a6569da1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc025fe8f2f944be83d8e52a94575238)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:46.037571 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd91403-b732-4add-935f-6f5a6569da1a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:46.056421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:46.097809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cfd91403-b732-4add-935f-6f5a6569da1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc025fe8f2f944be83d8e52a94575238', 'request_id': 'req-cfd91403-b732-4add-935f-6f5a6569da1a'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:46.105858 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cfd91403-b732-4add-935f-6f5a6569da1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc025fe8f2f944be83d8e52a94575238', 'request_id': 'req-cfd91403-b732-4add-935f-6f5a6569da1a'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:46.111865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cfd91403-b732-4add-935f-6f5a6569da1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc025fe8f2f944be83d8e52a94575238', 'request_id': 'req-cfd91403-b732-4add-935f-6f5a6569da1a'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:46.123716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cfd91403-b732-4add-935f-6f5a6569da1a admin admin] The token cache is being invalidate because project dc025fe8f2f944be83d8e52a94575238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:46.123936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cfd91403-b732-4add-935f-6f5a6569da1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:46.124596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cfd91403-b732-4add-935f-6f5a6569da1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:46.126439 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1122/2278] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:46 2022] DELETE /identity/v3/projects/dc025fe8f2f944be83d8e52a94575238 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:46.131403 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b91e-0fa7-41ea-9a4a-ec61881589db None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:46.131960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b91e-0fa7-41ea-9a4a-ec61881589db None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:46.131960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b91e-0fa7-41ea-9a4a-ec61881589db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:46.154005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2213b91e-0fa7-41ea-9a4a-ec61881589db None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:46.155334 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2213b91e-0fa7-41ea-9a4a-ec61881589db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:46.230236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1157/2279] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:46.238426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37d710-f771-42d1-886f-f0bb3bb4d65d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:46.238807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37d710-f771-42d1-886f-f0bb3bb4d65d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:46.239141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37d710-f771-42d1-886f-f0bb3bb4d65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:46.261086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-eb37d710-f771-42d1-886f-f0bb3bb4d65d None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:46.262399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-eb37d710-f771-42d1-886f-f0bb3bb4d65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:46.271258 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1123/2280] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:46.276114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213b91e-0fa7-41ea-9a4a-ec61881589db None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:46.286554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213b91e-0fa7-41ea-9a4a-ec61881589db None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:46.291401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe177c4-6db4-4710-8620-6758565abacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:46.292051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe177c4-6db4-4710-8620-6758565abacf admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:46.292331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe177c4-6db4-4710-8620-6758565abacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:46.292602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe177c4-6db4-4710-8620-6758565abacf admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:46.293253 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe177c4-6db4-4710-8620-6758565abacf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:41:46.293555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe177c4-6db4-4710-8620-6758565abacf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:46.295168 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe177c4-6db4-4710-8620-6758565abacf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:46.302268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:46.307719 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1158/2281] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:41:46.312461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37d710-f771-42d1-886f-f0bb3bb4d65d None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:46.322742 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37d710-f771-42d1-886f-f0bb3bb4d65d None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:46.328269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb3cac-3a7a-4bfe-84ca-d4d927e54fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:46.328991 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb3cac-3a7a-4bfe-84ca-d4d927e54fbb admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:46.329353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb3cac-3a7a-4bfe-84ca-d4d927e54fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:46.329745 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb3cac-3a7a-4bfe-84ca-d4d927e54fbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:46.330667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb3cac-3a7a-4bfe-84ca-d4d927e54fbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:46.332684 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb3cac-3a7a-4bfe-84ca-d4d927e54fbb admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:46.366849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1124/2282] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:46 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:41:46.368017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe177c4-6db4-4710-8620-6758565abacf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:46.378397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe177c4-6db4-4710-8620-6758565abacf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:46.439136 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514abc7e-bbeb-4d88-a917-aea16ad9bfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:46.439745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514abc7e-bbeb-4d88-a917-aea16ad9bfcc admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:46.440050 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514abc7e-bbeb-4d88-a917-aea16ad9bfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:46.440313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514abc7e-bbeb-4d88-a917-aea16ad9bfcc admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:46.440939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514abc7e-bbeb-4d88-a917-aea16ad9bfcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:46.442880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514abc7e-bbeb-4d88-a917-aea16ad9bfcc admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:46.466076 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1159/2283] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:46 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:46.470493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb3cac-3a7a-4bfe-84ca-d4d927e54fbb admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:46.481191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb3cac-3a7a-4bfe-84ca-d4d927e54fbb admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:46.486349 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17567c-6ff3-4bd6-8986-a6941efcf559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:46.487034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17567c-6ff3-4bd6-8986-a6941efcf559 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:46.487366 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17567c-6ff3-4bd6-8986-a6941efcf559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:46.487696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17567c-6ff3-4bd6-8986-a6941efcf559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:46.488443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17567c-6ff3-4bd6-8986-a6941efcf559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:46.490097 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17567c-6ff3-4bd6-8986-a6941efcf559 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:46.501188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1125/2284] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:46.505269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514abc7e-bbeb-4d88-a917-aea16ad9bfcc admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:46.516303 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514abc7e-bbeb-4d88-a917-aea16ad9bfcc admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:46.521241 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61068afc-a12c-41a0-a86a-656b91d88217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:46.525982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61068afc-a12c-41a0-a86a-656b91d88217 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:46.525982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61068afc-a12c-41a0-a86a-656b91d88217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:46.525982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61068afc-a12c-41a0-a86a-656b91d88217 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:46.525982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61068afc-a12c-41a0-a86a-656b91d88217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:46.525982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61068afc-a12c-41a0-a86a-656b91d88217 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:46.536386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1160/2285] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:46.541123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17567c-6ff3-4bd6-8986-a6941efcf559 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:46.551284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17567c-6ff3-4bd6-8986-a6941efcf559 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:46.558630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b6687-dc07-4666-a61c-9bd67fc130a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:46.559345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6687-dc07-4666-a61c-9bd67fc130a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:46.559646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6687-dc07-4666-a61c-9bd67fc130a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:46.559945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6687-dc07-4666-a61c-9bd67fc130a5 admin admin] PATH_INFO: `/v3/projects/a91bfdf2d4f84b718283328125a61832/users/73f3516727914c4aae7b9d85eaa4d035/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:46.578088 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b6687-dc07-4666-a61c-9bd67fc130a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a91bfdf2d4f84b718283328125a61832, user_id=73f3516727914c4aae7b9d85eaa4d035, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:46.580780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b6687-dc07-4666-a61c-9bd67fc130a5 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:46.596269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1126/2286] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:46 2022] PUT /identity/v3/projects/a91bfdf2d4f84b718283328125a61832/users/73f3516727914c4aae7b9d85eaa4d035/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:46.607273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e9e16-24ea-4f98-8ee5-1010f8ed2f19 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:46.607564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e9e16-24ea-4f98-8ee5-1010f8ed2f19 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:46.607826 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e9e16-24ea-4f98-8ee5-1010f8ed2f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:46.636838 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bb1e9e16-24ea-4f98-8ee5-1010f8ed2f19 None None] MFA Rules not processed for user `73f3516727914c4aae7b9d85eaa4d035`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:46.637975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bb1e9e16-24ea-4f98-8ee5-1010f8ed2f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:46.714702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1161/2287] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:46.719723 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b6687-dc07-4666-a61c-9bd67fc130a5 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:46.787756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b6687-dc07-4666-a61c-9bd67fc130a5 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:46.793059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea61ccc-f842-4bf0-b61a-a2b176cdaf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:46.793800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea61ccc-f842-4bf0-b61a-a2b176cdaf06 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:46.794104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea61ccc-f842-4bf0-b61a-a2b176cdaf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:46.794441 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea61ccc-f842-4bf0-b61a-a2b176cdaf06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:46.795079 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea61ccc-f842-4bf0-b61a-a2b176cdaf06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:46.797114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea61ccc-f842-4bf0-b61a-a2b176cdaf06 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:46.829536 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1127/2288] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:46 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:46.833715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e9e16-24ea-4f98-8ee5-1010f8ed2f19 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:46.846028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e9e16-24ea-4f98-8ee5-1010f8ed2f19 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:46.908356 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5865b9-19fb-4876-bb27-79eb3c67de08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:46.908900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5865b9-19fb-4876-bb27-79eb3c67de08 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:46.909163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5865b9-19fb-4876-bb27-79eb3c67de08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:46.909433 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5865b9-19fb-4876-bb27-79eb3c67de08 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:46.910066 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5865b9-19fb-4876-bb27-79eb3c67de08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:46.912046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5865b9-19fb-4876-bb27-79eb3c67de08 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:46.924181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:46.936972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1162/2289] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:46 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:41:46.941046 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea61ccc-f842-4bf0-b61a-a2b176cdaf06 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:46.951952 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea61ccc-f842-4bf0-b61a-a2b176cdaf06 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:46.956882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebefe39-f866-4b18-98a1-924eb5210a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:46.957515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebefe39-f866-4b18-98a1-924eb5210a28 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:46.957816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebefe39-f866-4b18-98a1-924eb5210a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:46.958089 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebefe39-f866-4b18-98a1-924eb5210a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:46.958747 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebefe39-f866-4b18-98a1-924eb5210a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:46.960495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebefe39-f866-4b18-98a1-924eb5210a28 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:46.972269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1128/2290] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:46.976900 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5865b9-19fb-4876-bb27-79eb3c67de08 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:46.987731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5865b9-19fb-4876-bb27-79eb3c67de08 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:46.993198 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad32f9-97f1-4d3c-87b4-3e5bef24c37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:46.993805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad32f9-97f1-4d3c-87b4-3e5bef24c37f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:46.994079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad32f9-97f1-4d3c-87b4-3e5bef24c37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:46.994345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad32f9-97f1-4d3c-87b4-3e5bef24c37f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:46.994975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad32f9-97f1-4d3c-87b4-3e5bef24c37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:46.996593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad32f9-97f1-4d3c-87b4-3e5bef24c37f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:47.007132 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1163/2291] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:47.011862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebefe39-f866-4b18-98a1-924eb5210a28 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:47.022019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebefe39-f866-4b18-98a1-924eb5210a28 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:47.026870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853e880-5322-4511-b9cd-5ed5d2c7dd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:47.027515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853e880-5322-4511-b9cd-5ed5d2c7dd34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:47.027796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853e880-5322-4511-b9cd-5ed5d2c7dd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:47.028100 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853e880-5322-4511-b9cd-5ed5d2c7dd34 admin admin] PATH_INFO: `/v3/projects/b6b07a5bdc0d402eb73fbe75e67aed96/users/bf468b7b9f154b95a761e80200043714/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:47.041031 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:47.044472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853e880-5322-4511-b9cd-5ed5d2c7dd34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6b07a5bdc0d402eb73fbe75e67aed96, user_id=bf468b7b9f154b95a761e80200043714, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:47.048189 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853e880-5322-4511-b9cd-5ed5d2c7dd34 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:47.063334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1129/2292] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:47 2022] PUT /identity/v3/projects/b6b07a5bdc0d402eb73fbe75e67aed96/users/bf468b7b9f154b95a761e80200043714/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:47.075482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba46b1-1d8e-48df-b336-b81a599f685d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:47.075759 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba46b1-1d8e-48df-b336-b81a599f685d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:47.076069 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba46b1-1d8e-48df-b336-b81a599f685d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:47.106387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-77ba46b1-1d8e-48df-b336-b81a599f685d None None] MFA Rules not processed for user `bf468b7b9f154b95a761e80200043714`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:47.107631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-77ba46b1-1d8e-48df-b336-b81a599f685d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:47.174322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1164/2293] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:47.179146 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853e880-5322-4511-b9cd-5ed5d2c7dd34 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:47.246924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853e880-5322-4511-b9cd-5ed5d2c7dd34 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:47.256260 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17643dbe-d99e-4825-8554-fb86c3156800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:47.257109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17643dbe-d99e-4825-8554-fb86c3156800 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:47.257396 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17643dbe-d99e-4825-8554-fb86c3156800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:47.257665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17643dbe-d99e-4825-8554-fb86c3156800 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:47.258302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17643dbe-d99e-4825-8554-fb86c3156800 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:47.260247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17643dbe-d99e-4825-8554-fb86c3156800 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:47.290071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1130/2294] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:47 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:47.294215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba46b1-1d8e-48df-b336-b81a599f685d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:47.306270 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba46b1-1d8e-48df-b336-b81a599f685d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:47.367475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3528f-72d5-438e-af65-856d274a1f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:47.368072 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3528f-72d5-438e-af65-856d274a1f0c admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:47.368350 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3528f-72d5-438e-af65-856d274a1f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:47.369029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3528f-72d5-438e-af65-856d274a1f0c admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:47.369246 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3528f-72d5-438e-af65-856d274a1f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:47.371155 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3528f-72d5-438e-af65-856d274a1f0c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:47.395097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1165/2295] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:47 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:41:47.399065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17643dbe-d99e-4825-8554-fb86c3156800 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:47.415162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17643dbe-d99e-4825-8554-fb86c3156800 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:47.421323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c90e7-992c-40f4-af1d-c7eee71d2472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:47.421764 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c90e7-992c-40f4-af1d-c7eee71d2472 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:47.422151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c90e7-992c-40f4-af1d-c7eee71d2472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:47.422568 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c90e7-992c-40f4-af1d-c7eee71d2472 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:47.424242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c90e7-992c-40f4-af1d-c7eee71d2472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:47.426206 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c90e7-992c-40f4-af1d-c7eee71d2472 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:47.439373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1131/2296] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:41:47.443799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3528f-72d5-438e-af65-856d274a1f0c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:47.454571 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3528f-72d5-438e-af65-856d274a1f0c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:47.459626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e835df3-7e34-4562-bedc-2974c55eb147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:47.460325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e835df3-7e34-4562-bedc-2974c55eb147 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:47.460599 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e835df3-7e34-4562-bedc-2974c55eb147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:47.460868 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e835df3-7e34-4562-bedc-2974c55eb147 admin admin] PATH_INFO: `/v3/projects/f603aaeee2f0405aaac19e56f4107f11/users/644469f9e1e4488d8140689a09865102/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:47.497928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e835df3-7e34-4562-bedc-2974c55eb147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f603aaeee2f0405aaac19e56f4107f11, user_id=644469f9e1e4488d8140689a09865102, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:47.500687 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e835df3-7e34-4562-bedc-2974c55eb147 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:47.514687 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1166/2297] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:47 2022] PUT /identity/v3/projects/f603aaeee2f0405aaac19e56f4107f11/users/644469f9e1e4488d8140689a09865102/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:47.518677 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c90e7-992c-40f4-af1d-c7eee71d2472 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:47.529165 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c90e7-992c-40f4-af1d-c7eee71d2472 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:47.591759 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be26847-c9a8-4d70-9998-e618a0bb7215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:47.592401 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be26847-c9a8-4d70-9998-e618a0bb7215 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:47.592669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be26847-c9a8-4d70-9998-e618a0bb7215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:47.592936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be26847-c9a8-4d70-9998-e618a0bb7215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:47.593559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be26847-c9a8-4d70-9998-e618a0bb7215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:47.595169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be26847-c9a8-4d70-9998-e618a0bb7215 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:47.604996 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1132/2298] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:47.610095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e835df3-7e34-4562-bedc-2974c55eb147 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:47.621416 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e835df3-7e34-4562-bedc-2974c55eb147 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:47.626876 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37e869-2c8a-4edc-8615-05baa90a61f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:47.627550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37e869-2c8a-4edc-8615-05baa90a61f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:47.627823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37e869-2c8a-4edc-8615-05baa90a61f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:47.628121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37e869-2c8a-4edc-8615-05baa90a61f8 admin admin] PATH_INFO: `/v3/projects/f603aaeee2f0405aaac19e56f4107f11/users/644469f9e1e4488d8140689a09865102/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:47.630077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37e869-2c8a-4edc-8615-05baa90a61f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f603aaeee2f0405aaac19e56f4107f11, user_id=644469f9e1e4488d8140689a09865102, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:47.632755 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37e869-2c8a-4edc-8615-05baa90a61f8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:47.644298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1167/2299] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:47 2022] PUT /identity/v3/projects/f603aaeee2f0405aaac19e56f4107f11/users/644469f9e1e4488d8140689a09865102/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:47.656652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed76b-d467-4496-99b6-bdb7e0a9d6d2 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:47.656944 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed76b-d467-4496-99b6-bdb7e0a9d6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:47.657270 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ed76b-d467-4496-99b6-bdb7e0a9d6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:47.687648 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-d01ed76b-d467-4496-99b6-bdb7e0a9d6d2 None None] MFA Rules not processed for user `644469f9e1e4488d8140689a09865102`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:47.688805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d01ed76b-d467-4496-99b6-bdb7e0a9d6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:47.738342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:47.765350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1133/2300] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:47.775437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37e869-2c8a-4edc-8615-05baa90a61f8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:47.777359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-df37e869-2c8a-4edc-8615-05baa90a61f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:47.810524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:47.843282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37e869-2c8a-4edc-8615-05baa90a61f8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:47.846921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084e160-8014-4406-bc69-05c31b999b05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:47.847526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084e160-8014-4406-bc69-05c31b999b05 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:47.847806 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084e160-8014-4406-bc69-05c31b999b05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:47.848128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084e160-8014-4406-bc69-05c31b999b05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:47.858026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084e160-8014-4406-bc69-05c31b999b05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:47.859464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084e160-8014-4406-bc69-05c31b999b05 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:47.869817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1168/2301] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:48.429747 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ed76b-d467-4496-99b6-bdb7e0a9d6d2 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:48.431136 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d01ed76b-d467-4496-99b6-bdb7e0a9d6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:48.443701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ed76b-d467-4496-99b6-bdb7e0a9d6d2 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:48.447909 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefca7e8-b1c4-4a1f-ac51-8c9b93f0cec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:48.448786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefca7e8-b1c4-4a1f-ac51-8c9b93f0cec7 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:48.449066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefca7e8-b1c4-4a1f-ac51-8c9b93f0cec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:48.449336 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefca7e8-b1c4-4a1f-ac51-8c9b93f0cec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:48.512610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefca7e8-b1c4-4a1f-ac51-8c9b93f0cec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:48.513723 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefca7e8-b1c4-4a1f-ac51-8c9b93f0cec7 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:48.527008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1134/2302] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:50.430897 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084e160-8014-4406-bc69-05c31b999b05 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:50.432646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9084e160-8014-4406-bc69-05c31b999b05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:50.494296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084e160-8014-4406-bc69-05c31b999b05 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:50.498458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba91f499-55a1-473c-aa0d-bba65cbf1d6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:50.499067 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91f499-55a1-473c-aa0d-bba65cbf1d6b service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:50.499342 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91f499-55a1-473c-aa0d-bba65cbf1d6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:50.499608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91f499-55a1-473c-aa0d-bba65cbf1d6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:50.501554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ba91f499-55a1-473c-aa0d-bba65cbf1d6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:50.512466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba91f499-55a1-473c-aa0d-bba65cbf1d6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:50.513890 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba91f499-55a1-473c-aa0d-bba65cbf1d6b service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:50.522379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1169/2303] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:52.662433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefca7e8-b1c4-4a1f-ac51-8c9b93f0cec7 service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:52.672046 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefca7e8-b1c4-4a1f-ac51-8c9b93f0cec7 service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:52.676424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12f732-ec1e-4232-9fbe-c84c8cb7ad11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:52.677034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12f732-ec1e-4232-9fbe-c84c8cb7ad11 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:52.677315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12f732-ec1e-4232-9fbe-c84c8cb7ad11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:52.677628 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12f732-ec1e-4232-9fbe-c84c8cb7ad11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:52.739011 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12f732-ec1e-4232-9fbe-c84c8cb7ad11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:52.740600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12f732-ec1e-4232-9fbe-c84c8cb7ad11 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:52.750295 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1135/2304] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:54.908829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba91f499-55a1-473c-aa0d-bba65cbf1d6b service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:54.985090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba91f499-55a1-473c-aa0d-bba65cbf1d6b service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:54.991203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1b5c4-2362-427e-a0ab-ce1c5b66bfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:54.991830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1b5c4-2362-427e-a0ab-ce1c5b66bfd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:54.992134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1b5c4-2362-427e-a0ab-ce1c5b66bfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:54.992399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1b5c4-2362-427e-a0ab-ce1c5b66bfd9 admin admin] PATH_INFO: `/v3/users/73f3516727914c4aae7b9d85eaa4d035` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:54.993546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1b5c4-2362-427e-a0ab-ce1c5b66bfd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73f3516727914c4aae7b9d85eaa4d035)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:54.995449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1b5c4-2362-427e-a0ab-ce1c5b66bfd9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:55.057503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e1b1b5c4-2362-427e-a0ab-ce1c5b66bfd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73f3516727914c4aae7b9d85eaa4d035'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.066182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e1b1b5c4-2362-427e-a0ab-ce1c5b66bfd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73f3516727914c4aae7b9d85eaa4d035'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.066828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e1b1b5c4-2362-427e-a0ab-ce1c5b66bfd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73f3516727914c4aae7b9d85eaa4d035'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.080333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e1b1b5c4-2362-427e-a0ab-ce1c5b66bfd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73f3516727914c4aae7b9d85eaa4d035'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.082275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:55.090580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e1b1b5c4-2362-427e-a0ab-ce1c5b66bfd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73f3516727914c4aae7b9d85eaa4d035'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.096974 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1170/2305] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:54 2022] DELETE /identity/v3/users/73f3516727914c4aae7b9d85eaa4d035 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:55.102076 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d1fed-1903-4941-8fb3-ad8d7d66f2cd None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:55.102549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d1fed-1903-4941-8fb3-ad8d7d66f2cd None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:55.102643 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d1fed-1903-4941-8fb3-ad8d7d66f2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:55.123822 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-0e7d1fed-1903-4941-8fb3-ad8d7d66f2cd None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:55.125035 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0e7d1fed-1903-4941-8fb3-ad8d7d66f2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:55.193474 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1136/2306] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:55.203934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1b5c4-2362-427e-a0ab-ce1c5b66bfd9 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:55.205652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e1b1b5c4-2362-427e-a0ab-ce1c5b66bfd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:55.274203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1b5c4-2362-427e-a0ab-ce1c5b66bfd9 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:55.281412 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058ffd1-a951-4533-95e6-73d6a1e0ac86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:55.282386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058ffd1-a951-4533-95e6-73d6a1e0ac86 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:55.282842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058ffd1-a951-4533-95e6-73d6a1e0ac86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:55.283299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058ffd1-a951-4533-95e6-73d6a1e0ac86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:55.296733 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058ffd1-a951-4533-95e6-73d6a1e0ac86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:55.298198 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058ffd1-a951-4533-95e6-73d6a1e0ac86 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:55.309191 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1171/2307] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:55.597945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d1fed-1903-4941-8fb3-ad8d7d66f2cd None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:55.600298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0e7d1fed-1903-4941-8fb3-ad8d7d66f2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:55.616407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d1fed-1903-4941-8fb3-ad8d7d66f2cd None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:55.621983 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:55.622633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:55.622904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:55.623411 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] PATH_INFO: `/v3/projects/a91bfdf2d4f84b718283328125a61832` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:55.624576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a91bfdf2d4f84b718283328125a61832)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:55.626509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:55.652857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:55.689579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a91bfdf2d4f84b718283328125a61832', 'request_id': 'req-82a558e2-cf43-40be-a568-10cd5b156a19'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.699108 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a91bfdf2d4f84b718283328125a61832', 'request_id': 'req-82a558e2-cf43-40be-a568-10cd5b156a19'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.705173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a91bfdf2d4f84b718283328125a61832', 'request_id': 'req-82a558e2-cf43-40be-a568-10cd5b156a19'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.717666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] The token cache is being invalidate because project a91bfdf2d4f84b718283328125a61832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:55.717866 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.718659 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.720339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1137/2308] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:55 2022] DELETE /identity/v3/projects/a91bfdf2d4f84b718283328125a61832 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:55.724601 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058ffd1-a951-4533-95e6-73d6a1e0ac86 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:55.735623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058ffd1-a951-4533-95e6-73d6a1e0ac86 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:55.801055 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff83d1-c30c-4a5a-a2c7-537d249ec590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:55.801695 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff83d1-c30c-4a5a-a2c7-537d249ec590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:55.801991 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff83d1-c30c-4a5a-a2c7-537d249ec590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:55.802255 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff83d1-c30c-4a5a-a2c7-537d249ec590 admin admin] PATH_INFO: `/v3/users/bf468b7b9f154b95a761e80200043714` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:55.803655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff83d1-c30c-4a5a-a2c7-537d249ec590 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf468b7b9f154b95a761e80200043714)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:55.805680 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff83d1-c30c-4a5a-a2c7-537d249ec590 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:55.856554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6eff83d1-c30c-4a5a-a2c7-537d249ec590 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf468b7b9f154b95a761e80200043714'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.866015 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6eff83d1-c30c-4a5a-a2c7-537d249ec590 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf468b7b9f154b95a761e80200043714'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.866669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6eff83d1-c30c-4a5a-a2c7-537d249ec590 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf468b7b9f154b95a761e80200043714'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.878651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6eff83d1-c30c-4a5a-a2c7-537d249ec590 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf468b7b9f154b95a761e80200043714'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.885508 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6eff83d1-c30c-4a5a-a2c7-537d249ec590 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf468b7b9f154b95a761e80200043714'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:55.891545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1172/2309] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:55 2022] DELETE /identity/v3/users/bf468b7b9f154b95a761e80200043714 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:56.185034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:56.186482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:56.201783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a558e2-cf43-40be-a568-10cd5b156a19 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:56.262612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:56.263162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:56.263445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:56.263687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] PATH_INFO: `/v3/projects/b6b07a5bdc0d402eb73fbe75e67aed96` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:56.264940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6b07a5bdc0d402eb73fbe75e67aed96)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:56.266947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:56.316263 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6b07a5bdc0d402eb73fbe75e67aed96', 'request_id': 'req-78ef5a07-b70c-4a7b-925e-4d3440256d00'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.326058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6b07a5bdc0d402eb73fbe75e67aed96', 'request_id': 'req-78ef5a07-b70c-4a7b-925e-4d3440256d00'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.333754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6b07a5bdc0d402eb73fbe75e67aed96', 'request_id': 'req-78ef5a07-b70c-4a7b-925e-4d3440256d00'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.346360 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] The token cache is being invalidate because project b6b07a5bdc0d402eb73fbe75e67aed96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:56.346646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.347288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.348814 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1138/2310] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:56 2022] DELETE /identity/v3/projects/b6b07a5bdc0d402eb73fbe75e67aed96 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:56.353606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff83d1-c30c-4a5a-a2c7-537d249ec590 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:56.355697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6eff83d1-c30c-4a5a-a2c7-537d249ec590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:56.372186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff83d1-c30c-4a5a-a2c7-537d249ec590 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:56.381345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:56.433646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb2813-f507-45ce-bf83-3566ecdbdc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:56.434283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2813-f507-45ce-bf83-3566ecdbdc95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:56.434553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2813-f507-45ce-bf83-3566ecdbdc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:56.434817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2813-f507-45ce-bf83-3566ecdbdc95 admin admin] PATH_INFO: `/v3/users/644469f9e1e4488d8140689a09865102` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:56.436146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb2813-f507-45ce-bf83-3566ecdbdc95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=644469f9e1e4488d8140689a09865102)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:56.438121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb2813-f507-45ce-bf83-3566ecdbdc95 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:56.486720 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fcdb2813-f507-45ce-bf83-3566ecdbdc95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644469f9e1e4488d8140689a09865102'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.494798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fcdb2813-f507-45ce-bf83-3566ecdbdc95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644469f9e1e4488d8140689a09865102'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.495421 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fcdb2813-f507-45ce-bf83-3566ecdbdc95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644469f9e1e4488d8140689a09865102'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.507359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fcdb2813-f507-45ce-bf83-3566ecdbdc95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644469f9e1e4488d8140689a09865102'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.514306 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fcdb2813-f507-45ce-bf83-3566ecdbdc95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644469f9e1e4488d8140689a09865102'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.520336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1173/2311] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:56 2022] DELETE /identity/v3/users/644469f9e1e4488d8140689a09865102 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:56.797750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:56.799285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:56.813917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef5a07-b70c-4a7b-925e-4d3440256d00 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:56.851798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:56.878054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:56.878692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:56.878966 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:56.879242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207 admin admin] PATH_INFO: `/v3/projects/f603aaeee2f0405aaac19e56f4107f11` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:56.880425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f603aaeee2f0405aaac19e56f4107f11)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:56.882514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:56.939181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f603aaeee2f0405aaac19e56f4107f11', 'request_id': 'req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.949588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f603aaeee2f0405aaac19e56f4107f11', 'request_id': 'req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.955372 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f603aaeee2f0405aaac19e56f4107f11', 'request_id': 'req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.966450 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207 admin admin] The token cache is being invalidate because project f603aaeee2f0405aaac19e56f4107f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:56.966735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.967367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7ccb8da0-e4b3-4a2c-a2ec-5b0e5eff5207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:56.968904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1139/2312] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:56 2022] DELETE /identity/v3/projects/f603aaeee2f0405aaac19e56f4107f11 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:56.975125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f005da-e0e2-475f-b19b-b6fb2204c21a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:56.975411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f005da-e0e2-475f-b19b-b6fb2204c21a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:56.975681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f005da-e0e2-475f-b19b-b6fb2204c21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:56.997149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-03f005da-e0e2-475f-b19b-b6fb2204c21a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:56.998359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-03f005da-e0e2-475f-b19b-b6fb2204c21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:57.075536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1174/2313] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:57.084130 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13faf3e-fede-4a35-a333-8f3dd1ab4e85 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:57.084409 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13faf3e-fede-4a35-a333-8f3dd1ab4e85 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:57.084674 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13faf3e-fede-4a35-a333-8f3dd1ab4e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:57.105682 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-c13faf3e-fede-4a35-a333-8f3dd1ab4e85 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:57.106887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c13faf3e-fede-4a35-a333-8f3dd1ab4e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:57.115402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1140/2314] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:57.119667 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f005da-e0e2-475f-b19b-b6fb2204c21a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:57.129872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f005da-e0e2-475f-b19b-b6fb2204c21a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:57.135019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8150d6a-1a2c-428a-a896-fbbee16c9fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:57.135723 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8150d6a-1a2c-428a-a896-fbbee16c9fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:57.136101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8150d6a-1a2c-428a-a896-fbbee16c9fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:57.136364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8150d6a-1a2c-428a-a896-fbbee16c9fb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:57.137014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8150d6a-1a2c-428a-a896-fbbee16c9fb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:41:57.137310 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8150d6a-1a2c-428a-a896-fbbee16c9fb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:57.138959 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8150d6a-1a2c-428a-a896-fbbee16c9fb5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:57.151667 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1175/2315] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:41:57.156275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13faf3e-fede-4a35-a333-8f3dd1ab4e85 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:57.166615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13faf3e-fede-4a35-a333-8f3dd1ab4e85 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:57.171768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7045a1-6782-4a2e-9df0-4c8f359d493d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:57.172412 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7045a1-6782-4a2e-9df0-4c8f359d493d admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:57.172690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7045a1-6782-4a2e-9df0-4c8f359d493d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:57.172974 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7045a1-6782-4a2e-9df0-4c8f359d493d admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:57.173605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7045a1-6782-4a2e-9df0-4c8f359d493d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:57.175586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7045a1-6782-4a2e-9df0-4c8f359d493d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:57.210968 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1141/2316] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:57 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:41:57.212204 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8150d6a-1a2c-428a-a896-fbbee16c9fb5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:57.222638 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8150d6a-1a2c-428a-a896-fbbee16c9fb5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:57.241014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:57.283759 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c2379-5dd7-4380-a090-009130719495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:57.284416 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c2379-5dd7-4380-a090-009130719495 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:57.284791 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c2379-5dd7-4380-a090-009130719495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:57.285031 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c2379-5dd7-4380-a090-009130719495 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:57.285596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c2379-5dd7-4380-a090-009130719495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:57.288673 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c2379-5dd7-4380-a090-009130719495 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:57.311564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1176/2317] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:57 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:57.317044 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7045a1-6782-4a2e-9df0-4c8f359d493d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:57.334579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7045a1-6782-4a2e-9df0-4c8f359d493d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:57.339871 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d034b6-2415-4ecf-8f58-c84b24419a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:57.340511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d034b6-2415-4ecf-8f58-c84b24419a9c admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:57.340778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d034b6-2415-4ecf-8f58-c84b24419a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:57.341037 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d034b6-2415-4ecf-8f58-c84b24419a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:57.341678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d034b6-2415-4ecf-8f58-c84b24419a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:57.343282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d034b6-2415-4ecf-8f58-c84b24419a9c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:57.355311 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1142/2318] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:41:57.359875 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c2379-5dd7-4380-a090-009130719495 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:57.371072 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c2379-5dd7-4380-a090-009130719495 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:57.377185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016e0b7-320d-44d8-a034-b39605904de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:57.377783 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016e0b7-320d-44d8-a034-b39605904de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:57.378058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016e0b7-320d-44d8-a034-b39605904de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:57.378319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016e0b7-320d-44d8-a034-b39605904de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:57.378947 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016e0b7-320d-44d8-a034-b39605904de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:57.380686 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016e0b7-320d-44d8-a034-b39605904de4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:57.394240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1177/2319] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:57.400007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d034b6-2415-4ecf-8f58-c84b24419a9c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:57.414727 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d034b6-2415-4ecf-8f58-c84b24419a9c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:57.423562 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9090412-758f-44ac-b638-c90cedfc5726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:57.424503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9090412-758f-44ac-b638-c90cedfc5726 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:57.424748 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9090412-758f-44ac-b638-c90cedfc5726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:57.425093 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9090412-758f-44ac-b638-c90cedfc5726 admin admin] PATH_INFO: `/v3/projects/58be4ad77f204248a254347db037c53a/users/bfc285831ae14c9da61e3ab5f7184fc8/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:57.442049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9090412-758f-44ac-b638-c90cedfc5726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58be4ad77f204248a254347db037c53a, user_id=bfc285831ae14c9da61e3ab5f7184fc8, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:57.444835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9090412-758f-44ac-b638-c90cedfc5726 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:57.461080 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1143/2320] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:57 2022] PUT /identity/v3/projects/58be4ad77f204248a254347db037c53a/users/bfc285831ae14c9da61e3ab5f7184fc8/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:57.474577 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea713285-b893-46c7-8197-ae788475d2e4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:57.474864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea713285-b893-46c7-8197-ae788475d2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:57.475126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea713285-b893-46c7-8197-ae788475d2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:57.505511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ea713285-b893-46c7-8197-ae788475d2e4 None None] MFA Rules not processed for user `bfc285831ae14c9da61e3ab5f7184fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:57.506751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ea713285-b893-46c7-8197-ae788475d2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:57.582023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1178/2321] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:57.587048 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9090412-758f-44ac-b638-c90cedfc5726 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:57.680425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9090412-758f-44ac-b638-c90cedfc5726 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:57.687073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754f0f5-4bc2-4ba1-a3ae-48eeb89bb597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:57.687990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754f0f5-4bc2-4ba1-a3ae-48eeb89bb597 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:57.688269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754f0f5-4bc2-4ba1-a3ae-48eeb89bb597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:57.688571 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754f0f5-4bc2-4ba1-a3ae-48eeb89bb597 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:57.689404 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754f0f5-4bc2-4ba1-a3ae-48eeb89bb597 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:57.691710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754f0f5-4bc2-4ba1-a3ae-48eeb89bb597 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:57.702794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:57.724783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1144/2322] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:57 2022] POST /identity/v3/projects => generated 389 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:57.728946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea713285-b893-46c7-8197-ae788475d2e4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:57.740244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea713285-b893-46c7-8197-ae788475d2e4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:57.808878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c610c76-94e1-46cc-ac05-016a23500212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:57.809499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c610c76-94e1-46cc-ac05-016a23500212 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:57.809776 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c610c76-94e1-46cc-ac05-016a23500212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:57.810044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c610c76-94e1-46cc-ac05-016a23500212 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:57.810704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c610c76-94e1-46cc-ac05-016a23500212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:57.812722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c610c76-94e1-46cc-ac05-016a23500212 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:57.834664 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1179/2323] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:57 2022] POST /identity/v3/users => generated 301 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:41:57.838952 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754f0f5-4bc2-4ba1-a3ae-48eeb89bb597 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:57.850482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754f0f5-4bc2-4ba1-a3ae-48eeb89bb597 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:57.855125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04d18c-28b5-4999-9c0f-9d316c76d11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:57.855739 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04d18c-28b5-4999-9c0f-9d316c76d11d admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:57.856053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04d18c-28b5-4999-9c0f-9d316c76d11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:57.856325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04d18c-28b5-4999-9c0f-9d316c76d11d admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:57.856960 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04d18c-28b5-4999-9c0f-9d316c76d11d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:57.858556 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04d18c-28b5-4999-9c0f-9d316c76d11d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:57.869882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1145/2324] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:57.874020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c610c76-94e1-46cc-ac05-016a23500212 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:57.886586 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c610c76-94e1-46cc-ac05-016a23500212 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:57.892563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cb7c8-25c5-4290-83ec-088a8f4e4221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:57.893259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cb7c8-25c5-4290-83ec-088a8f4e4221 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:57.893537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cb7c8-25c5-4290-83ec-088a8f4e4221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:57.893842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cb7c8-25c5-4290-83ec-088a8f4e4221 admin admin] PATH_INFO: `/v3/projects/d8b6bd50a4564cb59ad04ab34db5a107/users/f9b5a6d4bfd34569aa6516990210cae4/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:57.911584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cb7c8-25c5-4290-83ec-088a8f4e4221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b6bd50a4564cb59ad04ab34db5a107, user_id=f9b5a6d4bfd34569aa6516990210cae4, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:57.914759 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cb7c8-25c5-4290-83ec-088a8f4e4221 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:57.917314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:57.930736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1180/2325] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:57 2022] PUT /identity/v3/projects/d8b6bd50a4564cb59ad04ab34db5a107/users/f9b5a6d4bfd34569aa6516990210cae4/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:57.934964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04d18c-28b5-4999-9c0f-9d316c76d11d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:57.946651 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04d18c-28b5-4999-9c0f-9d316c76d11d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:57.992423 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8746e4-8adf-49d5-b338-a72ee4329ced None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:57.992704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8746e4-8adf-49d5-b338-a72ee4329ced None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:57.992993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8746e4-8adf-49d5-b338-a72ee4329ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:58.010814 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73899b06-db93-4ac8-ae8e-ba9be73237eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:58.011445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73899b06-db93-4ac8-ae8e-ba9be73237eb admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:58.011722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73899b06-db93-4ac8-ae8e-ba9be73237eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:58.012026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73899b06-db93-4ac8-ae8e-ba9be73237eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:58.012676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73899b06-db93-4ac8-ae8e-ba9be73237eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:58.013921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cc8746e4-8adf-49d5-b338-a72ee4329ced None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:58.014279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73899b06-db93-4ac8-ae8e-ba9be73237eb admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:58.015128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cc8746e4-8adf-49d5-b338-a72ee4329ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:58.028728 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1146/2326] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:58.029757 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73899b06-db93-4ac8-ae8e-ba9be73237eb admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:58.040101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73899b06-db93-4ac8-ae8e-ba9be73237eb admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:58.045103 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1e617-39fe-459e-9de9-beb158c09e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:58.045780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1e617-39fe-459e-9de9-beb158c09e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:58.046074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1e617-39fe-459e-9de9-beb158c09e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:58.046338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1e617-39fe-459e-9de9-beb158c09e45 admin admin] PATH_INFO: `/v3/projects/d8b6bd50a4564cb59ad04ab34db5a107/users/f9b5a6d4bfd34569aa6516990210cae4/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:58.048830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1e617-39fe-459e-9de9-beb158c09e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b6bd50a4564cb59ad04ab34db5a107, user_id=f9b5a6d4bfd34569aa6516990210cae4, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:58.051484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1e617-39fe-459e-9de9-beb158c09e45 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:58.063763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1147/2327] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:58 2022] PUT /identity/v3/projects/d8b6bd50a4564cb59ad04ab34db5a107/users/f9b5a6d4bfd34569aa6516990210cae4/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:58.073859 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1181/2328] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:41:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:58.076731 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24058a72-98c4-4058-a6ff-d2f4c057531b None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:58.077014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24058a72-98c4-4058-a6ff-d2f4c057531b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:58.077423 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24058a72-98c4-4058-a6ff-d2f4c057531b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:58.110085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-24058a72-98c4-4058-a6ff-d2f4c057531b None None] MFA Rules not processed for user `f9b5a6d4bfd34569aa6516990210cae4`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:58.111349 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-24058a72-98c4-4058-a6ff-d2f4c057531b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:58.123790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8746e4-8adf-49d5-b338-a72ee4329ced None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:58.124918 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cc8746e4-8adf-49d5-b338-a72ee4329ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:58.194639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1148/2329] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:58 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:58.207057 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24058a72-98c4-4058-a6ff-d2f4c057531b None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:58.208438 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8746e4-8adf-49d5-b338-a72ee4329ced None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:58.209145 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-24058a72-98c4-4058-a6ff-d2f4c057531b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:58.267981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:58.275058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24058a72-98c4-4058-a6ff-d2f4c057531b None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:58.277594 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb58d4-0846-4a72-b913-f6f58c6b172d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:58.278231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb58d4-0846-4a72-b913-f6f58c6b172d service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:58.278512 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb58d4-0846-4a72-b913-f6f58c6b172d service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:58.278782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb58d4-0846-4a72-b913-f6f58c6b172d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:58.280198 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d84c91-7ad8-4633-bc9a-50f873a1fcf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:58.280937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d84c91-7ad8-4633-bc9a-50f873a1fcf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:58.281307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d84c91-7ad8-4633-bc9a-50f873a1fcf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:58.281666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d84c91-7ad8-4633-bc9a-50f873a1fcf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:58.292490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d84c91-7ad8-4633-bc9a-50f873a1fcf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:58.294065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d84c91-7ad8-4633-bc9a-50f873a1fcf4 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:58.306249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1149/2330] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:58 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:58.326182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:58.360746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb58d4-0846-4a72-b913-f6f58c6b172d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:58.362360 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb58d4-0846-4a72-b913-f6f58c6b172d service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:58.379385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1182/2331] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:41:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 16:41:58.516469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d84c91-7ad8-4633-bc9a-50f873a1fcf4 service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:58.618325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d84c91-7ad8-4633-bc9a-50f873a1fcf4 service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:58.623642 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befdf0a-166b-4644-94c6-3771b83adde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:58.624632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befdf0a-166b-4644-94c6-3771b83adde4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:58.625111 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befdf0a-166b-4644-94c6-3771b83adde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:58.625840 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befdf0a-166b-4644-94c6-3771b83adde4 admin admin] PATH_INFO: `/v3/users/bfc285831ae14c9da61e3ab5f7184fc8` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:58.627175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befdf0a-166b-4644-94c6-3771b83adde4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc285831ae14c9da61e3ab5f7184fc8)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:58.632374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befdf0a-166b-4644-94c6-3771b83adde4 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:58.738713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1befdf0a-166b-4644-94c6-3771b83adde4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc285831ae14c9da61e3ab5f7184fc8'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:58.764791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1befdf0a-166b-4644-94c6-3771b83adde4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc285831ae14c9da61e3ab5f7184fc8'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:58.781960 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1befdf0a-166b-4644-94c6-3771b83adde4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc285831ae14c9da61e3ab5f7184fc8'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:58.790682 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1befdf0a-166b-4644-94c6-3771b83adde4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc285831ae14c9da61e3ab5f7184fc8'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:58.791469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-1befdf0a-166b-4644-94c6-3771b83adde4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc285831ae14c9da61e3ab5f7184fc8'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:58.800763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1150/2332] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:58 2022] DELETE /identity/v3/users/bfc285831ae14c9da61e3ab5f7184fc8 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:58.805270 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883570d4-8823-4bde-a510-a0a75e354439 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:58.805550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883570d4-8823-4bde-a510-a0a75e354439 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:58.805820 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883570d4-8823-4bde-a510-a0a75e354439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:58.832814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-883570d4-8823-4bde-a510-a0a75e354439 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:58.834129 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-883570d4-8823-4bde-a510-a0a75e354439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:58.920077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1183/2333] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:41:58.930243 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befdf0a-166b-4644-94c6-3771b83adde4 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:58.932289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1befdf0a-166b-4644-94c6-3771b83adde4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:59.000030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befdf0a-166b-4644-94c6-3771b83adde4 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:59.003935 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae71d69-c4ae-4fb2-aa0b-b93623311570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:59.004535 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae71d69-c4ae-4fb2-aa0b-b93623311570 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:59.004846 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae71d69-c4ae-4fb2-aa0b-b93623311570 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:59.005151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae71d69-c4ae-4fb2-aa0b-b93623311570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:59.016345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae71d69-c4ae-4fb2-aa0b-b93623311570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:59.017827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae71d69-c4ae-4fb2-aa0b-b93623311570 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:59.029137 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1151/2334] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:59.347716 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883570d4-8823-4bde-a510-a0a75e354439 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:59.349746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-883570d4-8823-4bde-a510-a0a75e354439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:59.372354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883570d4-8823-4bde-a510-a0a75e354439 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:59.388433 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:59.401760 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:59.401760 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:59.403955 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] PATH_INFO: `/v3/projects/58be4ad77f204248a254347db037c53a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:59.405370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58be4ad77f204248a254347db037c53a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:59.407383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:59.474111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58be4ad77f204248a254347db037c53a', 'request_id': 'req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:59.480790 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58be4ad77f204248a254347db037c53a', 'request_id': 'req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:59.490546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58be4ad77f204248a254347db037c53a', 'request_id': 'req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:59.505827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] The token cache is being invalidate because project 58be4ad77f204248a254347db037c53a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:59.506245 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:59.506796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:59.509060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1184/2335] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:59 2022] DELETE /identity/v3/projects/58be4ad77f204248a254347db037c53a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:59.513806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae71d69-c4ae-4fb2-aa0b-b93623311570 service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:59.534030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae71d69-c4ae-4fb2-aa0b-b93623311570 service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:59.547239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:59.631947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd4ce6-20f9-4f0e-9212-45772f47006b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:59.634376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd4ce6-20f9-4f0e-9212-45772f47006b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:59.637505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd4ce6-20f9-4f0e-9212-45772f47006b admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:59.638099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd4ce6-20f9-4f0e-9212-45772f47006b admin admin] PATH_INFO: `/v3/users/f9b5a6d4bfd34569aa6516990210cae4` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:59.649657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:59.651430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:59.657836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd4ce6-20f9-4f0e-9212-45772f47006b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9b5a6d4bfd34569aa6516990210cae4)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:59.659933 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd4ce6-20f9-4f0e-9212-45772f47006b admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:59.793254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2edd4ce6-20f9-4f0e-9212-45772f47006b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9b5a6d4bfd34569aa6516990210cae4'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:59.816927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:59.825949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2edd4ce6-20f9-4f0e-9212-45772f47006b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9b5a6d4bfd34569aa6516990210cae4'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:59.832007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e7e93-4421-4d90-a60d-d207c75fa7a7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:59.843591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2edd4ce6-20f9-4f0e-9212-45772f47006b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9b5a6d4bfd34569aa6516990210cae4'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:59.848117 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe1b89-732f-48a2-bcc7-4bbd5f0aea4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:59.849227 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe1b89-732f-48a2-bcc7-4bbd5f0aea4b service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:59.850673 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe1b89-732f-48a2-bcc7-4bbd5f0aea4b service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:59.851795 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe1b89-732f-48a2-bcc7-4bbd5f0aea4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:59.855500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-21fe1b89-732f-48a2-bcc7-4bbd5f0aea4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:59.859864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2edd4ce6-20f9-4f0e-9212-45772f47006b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9b5a6d4bfd34569aa6516990210cae4'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:59.861495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2edd4ce6-20f9-4f0e-9212-45772f47006b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9b5a6d4bfd34569aa6516990210cae4'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:59.876700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1152/2336] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:59 2022] DELETE /identity/v3/users/f9b5a6d4bfd34569aa6516990210cae4 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:59.942531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe1b89-732f-48a2-bcc7-4bbd5f0aea4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:59.944380 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe1b89-732f-48a2-bcc7-4bbd5f0aea4b service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:59.970412 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1185/2337] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:00.272723 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd4ce6-20f9-4f0e-9212-45772f47006b admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:00.274580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2edd4ce6-20f9-4f0e-9212-45772f47006b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:00.447789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd4ce6-20f9-4f0e-9212-45772f47006b admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:00.458627 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc592d9-fe81-4064-ada7-6542fa1db132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:00.461436 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc592d9-fe81-4064-ada7-6542fa1db132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:00.461752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc592d9-fe81-4064-ada7-6542fa1db132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:00.462045 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc592d9-fe81-4064-ada7-6542fa1db132 admin admin] PATH_INFO: `/v3/projects/d8b6bd50a4564cb59ad04ab34db5a107` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:00.464138 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc592d9-fe81-4064-ada7-6542fa1db132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b6bd50a4564cb59ad04ab34db5a107)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:00.466086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc592d9-fe81-4064-ada7-6542fa1db132 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:00.548692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:00.567842 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3cc592d9-fe81-4064-ada7-6542fa1db132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b6bd50a4564cb59ad04ab34db5a107', 'request_id': 'req-3cc592d9-fe81-4064-ada7-6542fa1db132'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:00.581306 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3cc592d9-fe81-4064-ada7-6542fa1db132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b6bd50a4564cb59ad04ab34db5a107', 'request_id': 'req-3cc592d9-fe81-4064-ada7-6542fa1db132'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:00.592296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3cc592d9-fe81-4064-ada7-6542fa1db132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b6bd50a4564cb59ad04ab34db5a107', 'request_id': 'req-3cc592d9-fe81-4064-ada7-6542fa1db132'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:00.606372 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3cc592d9-fe81-4064-ada7-6542fa1db132 admin admin] The token cache is being invalidate because project d8b6bd50a4564cb59ad04ab34db5a107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:00.606941 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3cc592d9-fe81-4064-ada7-6542fa1db132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:00.608019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3cc592d9-fe81-4064-ada7-6542fa1db132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:00.610130 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1153/2338] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:00 2022] DELETE /identity/v3/projects/d8b6bd50a4564cb59ad04ab34db5a107 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:00.617944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ad184-bc1e-4e1a-a96f-63f70845a713 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:00.618724 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ad184-bc1e-4e1a-a96f-63f70845a713 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:00.619310 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ad184-bc1e-4e1a-a96f-63f70845a713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:00.657200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-fa2ad184-bc1e-4e1a-a96f-63f70845a713 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:00.658959 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fa2ad184-bc1e-4e1a-a96f-63f70845a713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:00.745653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1186/2339] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:00.754786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62401ba5-2c8e-40ca-a052-efc12cddf915 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:00.755569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62401ba5-2c8e-40ca-a052-efc12cddf915 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:00.761200 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62401ba5-2c8e-40ca-a052-efc12cddf915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:00.786030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-62401ba5-2c8e-40ca-a052-efc12cddf915 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:00.787288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-62401ba5-2c8e-40ca-a052-efc12cddf915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:00.807457 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1154/2340] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:00.811761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ad184-bc1e-4e1a-a96f-63f70845a713 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:00.825390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ad184-bc1e-4e1a-a96f-63f70845a713 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:00.830599 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b865736-84d6-4469-8c08-b0368449be01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:00.831220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b865736-84d6-4469-8c08-b0368449be01 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:00.831500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b865736-84d6-4469-8c08-b0368449be01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:00.831775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b865736-84d6-4469-8c08-b0368449be01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:00.832466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b865736-84d6-4469-8c08-b0368449be01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:42:00.832783 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b865736-84d6-4469-8c08-b0368449be01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:00.834434 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b865736-84d6-4469-8c08-b0368449be01 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:00.850678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1187/2341] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:42:00.854845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62401ba5-2c8e-40ca-a052-efc12cddf915 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:00.874661 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62401ba5-2c8e-40ca-a052-efc12cddf915 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:00.879534 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530111f9-d6c5-4ea7-a6d5-1abd59b4ed3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:00.880506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530111f9-d6c5-4ea7-a6d5-1abd59b4ed3e admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:00.880843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530111f9-d6c5-4ea7-a6d5-1abd59b4ed3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:00.881186 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530111f9-d6c5-4ea7-a6d5-1abd59b4ed3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:00.881883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530111f9-d6c5-4ea7-a6d5-1abd59b4ed3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:00.886242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530111f9-d6c5-4ea7-a6d5-1abd59b4ed3e admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:00.922655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1155/2342] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:00 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:00.926730 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b865736-84d6-4469-8c08-b0368449be01 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:00.939971 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b865736-84d6-4469-8c08-b0368449be01 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:01.022309 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e869e1-f933-4913-aada-761e3a7450a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:01.023201 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e869e1-f933-4913-aada-761e3a7450a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:01.023704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e869e1-f933-4913-aada-761e3a7450a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:01.024298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e869e1-f933-4913-aada-761e3a7450a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:01.025549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e869e1-f933-4913-aada-761e3a7450a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:01.027611 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e869e1-f933-4913-aada-761e3a7450a5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:01.058697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1188/2343] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:00 2022] POST /identity/v3/users => generated 283 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:42:01.062969 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530111f9-d6c5-4ea7-a6d5-1abd59b4ed3e admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:01.074881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530111f9-d6c5-4ea7-a6d5-1abd59b4ed3e admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:01.079863 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c9b05-523c-4f4b-ab59-efadf0536986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:01.080595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c9b05-523c-4f4b-ab59-efadf0536986 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:01.080933 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c9b05-523c-4f4b-ab59-efadf0536986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:01.081298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c9b05-523c-4f4b-ab59-efadf0536986 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:01.082067 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c9b05-523c-4f4b-ab59-efadf0536986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:01.084306 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c9b05-523c-4f4b-ab59-efadf0536986 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:01.099055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1156/2344] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:01.100384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e869e1-f933-4913-aada-761e3a7450a5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:01.115509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e869e1-f933-4913-aada-761e3a7450a5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:01.124358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54776d-9611-469c-bca5-62ff4557cc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:01.124996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54776d-9611-469c-bca5-62ff4557cc20 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:01.125272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54776d-9611-469c-bca5-62ff4557cc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:01.125548 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54776d-9611-469c-bca5-62ff4557cc20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:01.126194 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54776d-9611-469c-bca5-62ff4557cc20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:01.127867 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54776d-9611-469c-bca5-62ff4557cc20 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:01.150322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1189/2345] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:01.152501 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c9b05-523c-4f4b-ab59-efadf0536986 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:01.174683 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c9b05-523c-4f4b-ab59-efadf0536986 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:01.185164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db5af2-63ad-449f-917e-4f896e7035ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:01.186260 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db5af2-63ad-449f-917e-4f896e7035ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:01.186752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db5af2-63ad-449f-917e-4f896e7035ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:01.187236 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db5af2-63ad-449f-917e-4f896e7035ca admin admin] PATH_INFO: `/v3/projects/9790f1d8bbe54bcc9927ecc25558864a/users/31aa437e662d4c6fbadd5612805ef123/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:01.214267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db5af2-63ad-449f-917e-4f896e7035ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9790f1d8bbe54bcc9927ecc25558864a, user_id=31aa437e662d4c6fbadd5612805ef123, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:01.218056 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db5af2-63ad-449f-917e-4f896e7035ca admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:01.243808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1157/2346] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:01 2022] PUT /identity/v3/projects/9790f1d8bbe54bcc9927ecc25558864a/users/31aa437e662d4c6fbadd5612805ef123/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:01.254382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166c134-5983-469d-98cd-d58ead5d20a6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:01.254664 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166c134-5983-469d-98cd-d58ead5d20a6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:01.254933 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166c134-5983-469d-98cd-d58ead5d20a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:01.298038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:01.331398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5166c134-5983-469d-98cd-d58ead5d20a6 None None] MFA Rules not processed for user `31aa437e662d4c6fbadd5612805ef123`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:01.334078 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5166c134-5983-469d-98cd-d58ead5d20a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:01.414491 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1190/2347] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:01.419055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db5af2-63ad-449f-917e-4f896e7035ca admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:01.498113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db5af2-63ad-449f-917e-4f896e7035ca admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:01.503457 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b377ab-445c-403f-bea3-e63ba8bebfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:01.504282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b377ab-445c-403f-bea3-e63ba8bebfb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:01.504561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b377ab-445c-403f-bea3-e63ba8bebfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:01.504858 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b377ab-445c-403f-bea3-e63ba8bebfb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:01.505903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b377ab-445c-403f-bea3-e63ba8bebfb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:01.508403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b377ab-445c-403f-bea3-e63ba8bebfb0 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:01.588266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1158/2348] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:01 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:01.588823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166c134-5983-469d-98cd-d58ead5d20a6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:01.593060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166c134-5983-469d-98cd-d58ead5d20a6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:01.698000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95a6ee-96cd-4dec-96fb-8a6d9a164c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:01.699013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a6ee-96cd-4dec-96fb-8a6d9a164c30 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:01.699201 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a6ee-96cd-4dec-96fb-8a6d9a164c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:01.699474 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95a6ee-96cd-4dec-96fb-8a6d9a164c30 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:01.700139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95a6ee-96cd-4dec-96fb-8a6d9a164c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:01.702235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95a6ee-96cd-4dec-96fb-8a6d9a164c30 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:01.740245 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1191/2349] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:01 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:01.744765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b377ab-445c-403f-bea3-e63ba8bebfb0 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:01.770174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b377ab-445c-403f-bea3-e63ba8bebfb0 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:01.775761 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71509b7-be40-4b99-9394-1c0723c9bef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:01.781248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71509b7-be40-4b99-9394-1c0723c9bef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:01.781826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71509b7-be40-4b99-9394-1c0723c9bef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:01.782163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71509b7-be40-4b99-9394-1c0723c9bef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:01.788511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71509b7-be40-4b99-9394-1c0723c9bef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:01.790319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71509b7-be40-4b99-9394-1c0723c9bef2 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:01.824972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1159/2350] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:42:01.825941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95a6ee-96cd-4dec-96fb-8a6d9a164c30 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:01.851436 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95a6ee-96cd-4dec-96fb-8a6d9a164c30 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:01.858305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5a8fb-13c3-470d-94d4-812e2f488bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:01.859140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5a8fb-13c3-470d-94d4-812e2f488bff admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:01.859544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5a8fb-13c3-470d-94d4-812e2f488bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:01.859919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5a8fb-13c3-470d-94d4-812e2f488bff admin admin] PATH_INFO: `/v3/projects/ac02847cda77423286cb00425d27634d/users/5b751a8beff04c7da3420da713575045/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:01.880274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5a8fb-13c3-470d-94d4-812e2f488bff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac02847cda77423286cb00425d27634d, user_id=5b751a8beff04c7da3420da713575045, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:01.883760 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5a8fb-13c3-470d-94d4-812e2f488bff admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:01.904300 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1192/2351] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:01 2022] PUT /identity/v3/projects/ac02847cda77423286cb00425d27634d/users/5b751a8beff04c7da3420da713575045/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:01.908438 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71509b7-be40-4b99-9394-1c0723c9bef2 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:01.928059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71509b7-be40-4b99-9394-1c0723c9bef2 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:01.991312 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:02.046775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada9606-1e9c-4774-a41c-e85b02dd9815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:02.047400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada9606-1e9c-4774-a41c-e85b02dd9815 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:02.047676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada9606-1e9c-4774-a41c-e85b02dd9815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:02.047985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada9606-1e9c-4774-a41c-e85b02dd9815 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:02.048849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada9606-1e9c-4774-a41c-e85b02dd9815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:02.050456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada9606-1e9c-4774-a41c-e85b02dd9815 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:02.063172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1160/2352] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:42:02.068768 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5a8fb-13c3-470d-94d4-812e2f488bff admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:02.080184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5a8fb-13c3-470d-94d4-812e2f488bff admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:02.085248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea547424-5cad-4b3b-b60d-1d19e1dedbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:02.085933 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea547424-5cad-4b3b-b60d-1d19e1dedbce admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:02.086213 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea547424-5cad-4b3b-b60d-1d19e1dedbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:02.086483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea547424-5cad-4b3b-b60d-1d19e1dedbce admin admin] PATH_INFO: `/v3/projects/ac02847cda77423286cb00425d27634d/users/5b751a8beff04c7da3420da713575045/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:02.088490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea547424-5cad-4b3b-b60d-1d19e1dedbce admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac02847cda77423286cb00425d27634d, user_id=5b751a8beff04c7da3420da713575045, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:02.092716 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea547424-5cad-4b3b-b60d-1d19e1dedbce admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:02.116582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1193/2353] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:02 2022] PUT /identity/v3/projects/ac02847cda77423286cb00425d27634d/users/5b751a8beff04c7da3420da713575045/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:42:02.118341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c518-c518-4768-b050-454373da66e8 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:02.118640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c518-c518-4768-b050-454373da66e8 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:02.118919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c518-c518-4768-b050-454373da66e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:02.154538 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-5dc4c518-c518-4768-b050-454373da66e8 None None] MFA Rules not processed for user `5b751a8beff04c7da3420da713575045`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:02.155778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5dc4c518-c518-4768-b050-454373da66e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:02.239192 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1161/2354] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:02.247221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea547424-5cad-4b3b-b60d-1d19e1dedbce admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:02.249171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ea547424-5cad-4b3b-b60d-1d19e1dedbce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:02.316669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea547424-5cad-4b3b-b60d-1d19e1dedbce admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:02.320714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15e21-30c9-40a1-bfbf-1ba04b3bb258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:02.321805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15e21-30c9-40a1-bfbf-1ba04b3bb258 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:02.322226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15e21-30c9-40a1-bfbf-1ba04b3bb258 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:02.322588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15e21-30c9-40a1-bfbf-1ba04b3bb258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:02.333742 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:02.338096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d15e21-30c9-40a1-bfbf-1ba04b3bb258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:02.340628 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d15e21-30c9-40a1-bfbf-1ba04b3bb258 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:02.357091 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1194/2355] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:02.490848 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4c518-c518-4768-b050-454373da66e8 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:02.492769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5dc4c518-c518-4768-b050-454373da66e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:02.507660 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4c518-c518-4768-b050-454373da66e8 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:02.512949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e162b9-2bd4-4994-851b-2a38e58101b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:02.513573 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e162b9-2bd4-4994-851b-2a38e58101b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:02.513841 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e162b9-2bd4-4994-851b-2a38e58101b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:02.514106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e162b9-2bd4-4994-851b-2a38e58101b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:02.579350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e162b9-2bd4-4994-851b-2a38e58101b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:02.581047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e162b9-2bd4-4994-851b-2a38e58101b3 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:02.592563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1162/2356] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:07.849010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15e21-30c9-40a1-bfbf-1ba04b3bb258 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:07.944367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15e21-30c9-40a1-bfbf-1ba04b3bb258 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:07.955079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21838aeb-d1fb-45c0-aac7-1c6e10a71031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:07.957477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21838aeb-d1fb-45c0-aac7-1c6e10a71031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:07.957788 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21838aeb-d1fb-45c0-aac7-1c6e10a71031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:07.958191 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21838aeb-d1fb-45c0-aac7-1c6e10a71031 admin admin] PATH_INFO: `/v3/users/31aa437e662d4c6fbadd5612805ef123` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:07.959911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21838aeb-d1fb-45c0-aac7-1c6e10a71031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31aa437e662d4c6fbadd5612805ef123)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:07.961944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21838aeb-d1fb-45c0-aac7-1c6e10a71031 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:08.017137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-21838aeb-d1fb-45c0-aac7-1c6e10a71031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31aa437e662d4c6fbadd5612805ef123'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.026646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-21838aeb-d1fb-45c0-aac7-1c6e10a71031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31aa437e662d4c6fbadd5612805ef123'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.027779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-21838aeb-d1fb-45c0-aac7-1c6e10a71031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31aa437e662d4c6fbadd5612805ef123'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.057171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-21838aeb-d1fb-45c0-aac7-1c6e10a71031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31aa437e662d4c6fbadd5612805ef123'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.057171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-21838aeb-d1fb-45c0-aac7-1c6e10a71031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31aa437e662d4c6fbadd5612805ef123'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.059936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1195/2357] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:07 2022] DELETE /identity/v3/users/31aa437e662d4c6fbadd5612805ef123 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:42:08.063350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379af67b-126e-4cf6-994d-86153586300e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:08.063842 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379af67b-126e-4cf6-994d-86153586300e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:08.063985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379af67b-126e-4cf6-994d-86153586300e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:08.086992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-379af67b-126e-4cf6-994d-86153586300e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:08.088375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-379af67b-126e-4cf6-994d-86153586300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:08.139780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:08.157917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1163/2358] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:42:08.169623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21838aeb-d1fb-45c0-aac7-1c6e10a71031 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:08.171150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-21838aeb-d1fb-45c0-aac7-1c6e10a71031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:08.233777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21838aeb-d1fb-45c0-aac7-1c6e10a71031 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:08.237596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea125f7-796f-4abd-a40d-5b55a6883eb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:08.238148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea125f7-796f-4abd-a40d-5b55a6883eb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:08.238418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea125f7-796f-4abd-a40d-5b55a6883eb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:08.238684 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea125f7-796f-4abd-a40d-5b55a6883eb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:08.247853 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea125f7-796f-4abd-a40d-5b55a6883eb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:08.249286 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea125f7-796f-4abd-a40d-5b55a6883eb7 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:08.259298 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1196/2359] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:08.585243 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379af67b-126e-4cf6-994d-86153586300e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:08.589289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-379af67b-126e-4cf6-994d-86153586300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:08.606196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379af67b-126e-4cf6-994d-86153586300e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:08.612236 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:08.613035 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:08.613347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:08.613621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] PATH_INFO: `/v3/projects/9790f1d8bbe54bcc9927ecc25558864a` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:08.615804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9790f1d8bbe54bcc9927ecc25558864a)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:08.618071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:08.687694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9790f1d8bbe54bcc9927ecc25558864a', 'request_id': 'req-d37de64c-2744-4cef-a6cd-a5fb67878413'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.703106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9790f1d8bbe54bcc9927ecc25558864a', 'request_id': 'req-d37de64c-2744-4cef-a6cd-a5fb67878413'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.721325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9790f1d8bbe54bcc9927ecc25558864a', 'request_id': 'req-d37de64c-2744-4cef-a6cd-a5fb67878413'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.750959 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] The token cache is being invalidate because project 9790f1d8bbe54bcc9927ecc25558864a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:08.751251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.754466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.756941 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1164/2360] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:08 2022] DELETE /identity/v3/projects/9790f1d8bbe54bcc9927ecc25558864a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:08.762113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea125f7-796f-4abd-a40d-5b55a6883eb7 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:08.772460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea125f7-796f-4abd-a40d-5b55a6883eb7 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:08.784315 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:08.832772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d21c0b-6aff-49d7-b74e-08ae97d18140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:08.833733 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d21c0b-6aff-49d7-b74e-08ae97d18140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:08.833999 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d21c0b-6aff-49d7-b74e-08ae97d18140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:08.834263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d21c0b-6aff-49d7-b74e-08ae97d18140 admin admin] PATH_INFO: `/v3/users/5b751a8beff04c7da3420da713575045` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:08.835415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d21c0b-6aff-49d7-b74e-08ae97d18140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b751a8beff04c7da3420da713575045)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:08.837351 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d21c0b-6aff-49d7-b74e-08ae97d18140 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:08.891794 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-57d21c0b-6aff-49d7-b74e-08ae97d18140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b751a8beff04c7da3420da713575045'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.900365 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-57d21c0b-6aff-49d7-b74e-08ae97d18140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b751a8beff04c7da3420da713575045'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.901087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-57d21c0b-6aff-49d7-b74e-08ae97d18140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b751a8beff04c7da3420da713575045'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.915148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-57d21c0b-6aff-49d7-b74e-08ae97d18140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b751a8beff04c7da3420da713575045'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.922472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-57d21c0b-6aff-49d7-b74e-08ae97d18140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b751a8beff04c7da3420da713575045'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:08.929194 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1197/2361] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:08 2022] DELETE /identity/v3/users/5b751a8beff04c7da3420da713575045 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:09.244292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:09.245755 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:09.266525 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37de64c-2744-4cef-a6cd-a5fb67878413 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:09.330497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:09.331140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:09.331413 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:09.331680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef admin admin] PATH_INFO: `/v3/projects/ac02847cda77423286cb00425d27634d` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:09.332846 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac02847cda77423286cb00425d27634d)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:09.335275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:09.386635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac02847cda77423286cb00425d27634d', 'request_id': 'req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:09.400193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac02847cda77423286cb00425d27634d', 'request_id': 'req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:09.406111 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac02847cda77423286cb00425d27634d', 'request_id': 'req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:09.417725 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef admin admin] The token cache is being invalidate because project ac02847cda77423286cb00425d27634d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:09.418016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:09.418673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9ca9a01a-cdec-4728-950f-5a31dd80d5ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:09.420241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1165/2362] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:09 2022] DELETE /identity/v3/projects/ac02847cda77423286cb00425d27634d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:09.426090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ad9bb-0c51-45cf-b377-b7e39fbdeb57 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:09.426450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ad9bb-0c51-45cf-b377-b7e39fbdeb57 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:09.426990 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ad9bb-0c51-45cf-b377-b7e39fbdeb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:09.448968 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2d2ad9bb-0c51-45cf-b377-b7e39fbdeb57 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:09.450147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2d2ad9bb-0c51-45cf-b377-b7e39fbdeb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:09.518100 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1198/2363] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:42:09.529804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127ea52-d916-4d48-85b7-621b84ac1c21 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:09.530086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127ea52-d916-4d48-85b7-621b84ac1c21 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:09.530367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127ea52-d916-4d48-85b7-621b84ac1c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:09.540648 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:09.552942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6127ea52-d916-4d48-85b7-621b84ac1c21 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:09.554243 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6127ea52-d916-4d48-85b7-621b84ac1c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:09.563927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1166/2364] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:09.568437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ad9bb-0c51-45cf-b377-b7e39fbdeb57 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:09.579713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ad9bb-0c51-45cf-b377-b7e39fbdeb57 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:09.586429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d671899-6e83-4630-986c-358e3096df99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:09.587053 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d671899-6e83-4630-986c-358e3096df99 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:09.587329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d671899-6e83-4630-986c-358e3096df99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:09.587592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d671899-6e83-4630-986c-358e3096df99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:09.588437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d671899-6e83-4630-986c-358e3096df99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:42:09.588742 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d671899-6e83-4630-986c-358e3096df99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:09.590473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d671899-6e83-4630-986c-358e3096df99 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:09.604639 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1199/2365] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:42:09.609652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127ea52-d916-4d48-85b7-621b84ac1c21 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:09.624073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127ea52-d916-4d48-85b7-621b84ac1c21 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:09.632218 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702222d-956e-4f2c-9b9c-95ed8578b632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:09.632849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702222d-956e-4f2c-9b9c-95ed8578b632 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:09.633124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702222d-956e-4f2c-9b9c-95ed8578b632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:09.633387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702222d-956e-4f2c-9b9c-95ed8578b632 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:09.634036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702222d-956e-4f2c-9b9c-95ed8578b632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:09.635961 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702222d-956e-4f2c-9b9c-95ed8578b632 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:09.669056 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1167/2366] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:09 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:09.675830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d671899-6e83-4630-986c-358e3096df99 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:09.694199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d671899-6e83-4630-986c-358e3096df99 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:09.716050 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:09.769244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222eba21-8fc1-4052-a856-3f2dbfd235b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:09.769862 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222eba21-8fc1-4052-a856-3f2dbfd235b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:09.770223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222eba21-8fc1-4052-a856-3f2dbfd235b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:09.770387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222eba21-8fc1-4052-a856-3f2dbfd235b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:09.771608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222eba21-8fc1-4052-a856-3f2dbfd235b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:09.773527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222eba21-8fc1-4052-a856-3f2dbfd235b5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:09.800026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1200/2367] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:09 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:09.801327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702222d-956e-4f2c-9b9c-95ed8578b632 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:09.812430 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702222d-956e-4f2c-9b9c-95ed8578b632 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:09.818281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3322ced-744e-4a6e-850a-4115216c3e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:09.819070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3322ced-744e-4a6e-850a-4115216c3e98 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:09.819355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3322ced-744e-4a6e-850a-4115216c3e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:09.819627 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3322ced-744e-4a6e-850a-4115216c3e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:09.820319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3322ced-744e-4a6e-850a-4115216c3e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:09.821952 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3322ced-744e-4a6e-850a-4115216c3e98 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:09.833395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1168/2368] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:09.838419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222eba21-8fc1-4052-a856-3f2dbfd235b5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:09.849558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222eba21-8fc1-4052-a856-3f2dbfd235b5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:09.854759 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ac896-4ecc-44a8-b2e4-ff77fdc32112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:09.855374 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ac896-4ecc-44a8-b2e4-ff77fdc32112 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:09.855797 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ac896-4ecc-44a8-b2e4-ff77fdc32112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:09.856781 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ac896-4ecc-44a8-b2e4-ff77fdc32112 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:09.857480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ac896-4ecc-44a8-b2e4-ff77fdc32112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:09.859583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ac896-4ecc-44a8-b2e4-ff77fdc32112 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:09.873162 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1201/2369] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:09.877508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3322ced-744e-4a6e-850a-4115216c3e98 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:09.888354 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3322ced-744e-4a6e-850a-4115216c3e98 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:09.893756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe96f0-5066-44ff-92a4-4337c2874849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:09.894471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe96f0-5066-44ff-92a4-4337c2874849 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:09.894753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe96f0-5066-44ff-92a4-4337c2874849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:09.895021 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe96f0-5066-44ff-92a4-4337c2874849 admin admin] PATH_INFO: `/v3/projects/3ac0e2a7b1434499a4c967e3b6dc101c/users/05d6331569784c82a534154b135a0a89/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:09.912946 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe96f0-5066-44ff-92a4-4337c2874849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ac0e2a7b1434499a4c967e3b6dc101c, user_id=05d6331569784c82a534154b135a0a89, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:09.916004 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe96f0-5066-44ff-92a4-4337c2874849 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:09.930064 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1169/2370] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:09 2022] PUT /identity/v3/projects/3ac0e2a7b1434499a4c967e3b6dc101c/users/05d6331569784c82a534154b135a0a89/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:09.940941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d1a0e-977d-4655-9e6b-3463633531ec None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:09.941435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d1a0e-977d-4655-9e6b-3463633531ec None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:09.941721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d1a0e-977d-4655-9e6b-3463633531ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:09.970591 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5e5d1a0e-977d-4655-9e6b-3463633531ec None None] MFA Rules not processed for user `05d6331569784c82a534154b135a0a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:09.971711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5e5d1a0e-977d-4655-9e6b-3463633531ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:10.041035 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1202/2371] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:42:10.054740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe96f0-5066-44ff-92a4-4337c2874849 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:10.056710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5cbe96f0-5066-44ff-92a4-4337c2874849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:10.119197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe96f0-5066-44ff-92a4-4337c2874849 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:10.123044 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b5f91-5b58-4a84-9765-88a2310769b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfe36a14a11493397e4018b773d3cc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:10.123664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b5f91-5b58-4a84-9765-88a2310769b6 service glance] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:10.123975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b5f91-5b58-4a84-9765-88a2310769b6 service glance] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:10.124249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b5f91-5b58-4a84-9765-88a2310769b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:10.133747 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b5f91-5b58-4a84-9765-88a2310769b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:10.135235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b5f91-5b58-4a84-9765-88a2310769b6 service glance] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:10.144421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1170/2372] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:12.298884 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d1a0e-977d-4655-9e6b-3463633531ec None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:12.301613 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5e5d1a0e-977d-4655-9e6b-3463633531ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:12.407742 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d1a0e-977d-4655-9e6b-3463633531ec None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:12.414205 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b005a-b3ab-4909-8f54-a85f62d56aff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:12.415097 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b005a-b3ab-4909-8f54-a85f62d56aff service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:12.415316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b005a-b3ab-4909-8f54-a85f62d56aff service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:12.415605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b005a-b3ab-4909-8f54-a85f62d56aff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:12.417493 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2a5b005a-b3ab-4909-8f54-a85f62d56aff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:12.431015 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b005a-b3ab-4909-8f54-a85f62d56aff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:12.432155 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b005a-b3ab-4909-8f54-a85f62d56aff service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:12.445272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1203/2373] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:13.347600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3bcd2-0ff0-417c-bdee-ad86b14e17ab None None] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:13.348182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3bcd2-0ff0-417c-bdee-ad86b14e17ab None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:13.348302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3bcd2-0ff0-417c-bdee-ad86b14e17ab None None] PATH_INFO: `/` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:13.350020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1171/2374] 10.222.0.62 () {54 vars in 921 bytes} [Thu Sep 1 16:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:42:13.355188 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e4423-79ed-4d06-9815-132e9e9d73dc None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:13.355718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e4423-79ed-4d06-9815-132e9e9d73dc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:13.355841 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e4423-79ed-4d06-9815-132e9e9d73dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:13.376825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-056e4423-79ed-4d06-9815-132e9e9d73dc None None] MFA Rules not processed for user `2912c02ff6db4cbc97c80cd923238136`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:13.378058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-056e4423-79ed-4d06-9815-132e9e9d73dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:13.391462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1204/2375] 10.222.0.62 () {58 vars in 1018 bytes} [Thu Sep 1 16:42:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:13.404529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3bcd2-0ff0-417c-bdee-ad86b14e17ab None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:13.419189 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3bcd2-0ff0-417c-bdee-ad86b14e17ab None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:13.426386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d126ee-0777-4f79-97a5-9818f7b291d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfe36a14a11493397e4018b773d3cc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:13.426386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d126ee-0777-4f79-97a5-9818f7b291d6 service glance] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:13.426386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d126ee-0777-4f79-97a5-9818f7b291d6 service glance] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:13.426386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d126ee-0777-4f79-97a5-9818f7b291d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:13.437245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d126ee-0777-4f79-97a5-9818f7b291d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:13.438691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d126ee-0777-4f79-97a5-9818f7b291d6 service glance] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:13.450900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1172/2376] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:15.488550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea269221-8c1c-4041-9fa6-04a10e07dc52 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:15.489095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea269221-8c1c-4041-9fa6-04a10e07dc52 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:15.489095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea269221-8c1c-4041-9fa6-04a10e07dc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:15.521044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ea269221-8c1c-4041-9fa6-04a10e07dc52 None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:15.522277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ea269221-8c1c-4041-9fa6-04a10e07dc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:15.526960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:15.592110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1205/2377] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:42:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:15.606042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d126ee-0777-4f79-97a5-9818f7b291d6 service glance] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:15.607588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b0d126ee-0777-4f79-97a5-9818f7b291d6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:15.680851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d126ee-0777-4f79-97a5-9818f7b291d6 service glance] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:15.685949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9d5d6-ff8b-4ac9-ac73-cc2eab978800 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:15.686570 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9d5d6-ff8b-4ac9-ac73-cc2eab978800 service swift] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:15.686840 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9d5d6-ff8b-4ac9-ac73-cc2eab978800 service swift] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:15.687104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9d5d6-ff8b-4ac9-ac73-cc2eab978800 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:15.695813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9d5d6-ff8b-4ac9-ac73-cc2eab978800 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:15.697472 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9d5d6-ff8b-4ac9-ac73-cc2eab978800 service swift] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:15.706050 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1173/2378] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:42:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:42:25.583844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=21 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:25.702875 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:28.014063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea269221-8c1c-4041-9fa6-04a10e07dc52 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:28.086890 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea269221-8c1c-4041-9fa6-04a10e07dc52 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:28.092502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce733f6a-cbe5-4863-86a4-d5881d85e186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:28.093173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce733f6a-cbe5-4863-86a4-d5881d85e186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:28.093480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce733f6a-cbe5-4863-86a4-d5881d85e186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:28.093787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce733f6a-cbe5-4863-86a4-d5881d85e186 admin admin] PATH_INFO: `/v3/users/05d6331569784c82a534154b135a0a89` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:28.095136 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce733f6a-cbe5-4863-86a4-d5881d85e186 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05d6331569784c82a534154b135a0a89)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:28.097297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce733f6a-cbe5-4863-86a4-d5881d85e186 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:28.150373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ce733f6a-cbe5-4863-86a4-d5881d85e186 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05d6331569784c82a534154b135a0a89'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:28.159017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ce733f6a-cbe5-4863-86a4-d5881d85e186 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05d6331569784c82a534154b135a0a89'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:28.159630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ce733f6a-cbe5-4863-86a4-d5881d85e186 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05d6331569784c82a534154b135a0a89'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:28.173398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ce733f6a-cbe5-4863-86a4-d5881d85e186 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05d6331569784c82a534154b135a0a89'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:28.182511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ce733f6a-cbe5-4863-86a4-d5881d85e186 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05d6331569784c82a534154b135a0a89'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:28.189871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1206/2379] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:28 2022] DELETE /identity/v3/users/05d6331569784c82a534154b135a0a89 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:28.194923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a60db5-eb4e-4f4e-91a9-d56df630a711 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:28.195468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a60db5-eb4e-4f4e-91a9-d56df630a711 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:28.195646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a60db5-eb4e-4f4e-91a9-d56df630a711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:28.217853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-c5a60db5-eb4e-4f4e-91a9-d56df630a711 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:28.219213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c5a60db5-eb4e-4f4e-91a9-d56df630a711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:28.279609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1174/2380] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:28.289579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce733f6a-cbe5-4863-86a4-d5881d85e186 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:28.291146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ce733f6a-cbe5-4863-86a4-d5881d85e186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:28.353873 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce733f6a-cbe5-4863-86a4-d5881d85e186 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:28.359058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb008fff-1f72-41bc-a494-a16e5b594b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:28.359677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb008fff-1f72-41bc-a494-a16e5b594b22 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:28.359977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb008fff-1f72-41bc-a494-a16e5b594b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:28.360248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb008fff-1f72-41bc-a494-a16e5b594b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:28.371052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb008fff-1f72-41bc-a494-a16e5b594b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:28.372594 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb008fff-1f72-41bc-a494-a16e5b594b22 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:28.383690 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1207/2381] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:28.680585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a60db5-eb4e-4f4e-91a9-d56df630a711 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:28.682298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c5a60db5-eb4e-4f4e-91a9-d56df630a711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:28.699875 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a60db5-eb4e-4f4e-91a9-d56df630a711 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:28.704717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f29ccc-a96f-472c-a42b-802c39e9ac7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:28.705399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f29ccc-a96f-472c-a42b-802c39e9ac7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:28.705832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f29ccc-a96f-472c-a42b-802c39e9ac7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:28.705936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f29ccc-a96f-472c-a42b-802c39e9ac7d admin admin] PATH_INFO: `/v3/projects/3ac0e2a7b1434499a4c967e3b6dc101c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:28.707179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f29ccc-a96f-472c-a42b-802c39e9ac7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ac0e2a7b1434499a4c967e3b6dc101c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:28.709173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f29ccc-a96f-472c-a42b-802c39e9ac7d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:28.781038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-28f29ccc-a96f-472c-a42b-802c39e9ac7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ac0e2a7b1434499a4c967e3b6dc101c', 'request_id': 'req-28f29ccc-a96f-472c-a42b-802c39e9ac7d'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:28.789299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-28f29ccc-a96f-472c-a42b-802c39e9ac7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ac0e2a7b1434499a4c967e3b6dc101c', 'request_id': 'req-28f29ccc-a96f-472c-a42b-802c39e9ac7d'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:28.796183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-28f29ccc-a96f-472c-a42b-802c39e9ac7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ac0e2a7b1434499a4c967e3b6dc101c', 'request_id': 'req-28f29ccc-a96f-472c-a42b-802c39e9ac7d'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:28.807807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-28f29ccc-a96f-472c-a42b-802c39e9ac7d admin admin] The token cache is being invalidate because project 3ac0e2a7b1434499a4c967e3b6dc101c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:28.808236 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-28f29ccc-a96f-472c-a42b-802c39e9ac7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:28.809040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-28f29ccc-a96f-472c-a42b-802c39e9ac7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:28.810638 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1175/2382] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:28 2022] DELETE /identity/v3/projects/3ac0e2a7b1434499a4c967e3b6dc101c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:28.816039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6d7c1-7920-4c8a-93b2-bbfca599f34f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:28.816314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6d7c1-7920-4c8a-93b2-bbfca599f34f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:28.816583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6d7c1-7920-4c8a-93b2-bbfca599f34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:28.837904 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5ed6d7c1-7920-4c8a-93b2-bbfca599f34f None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:28.839124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5ed6d7c1-7920-4c8a-93b2-bbfca599f34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:28.865254 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:28.913182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1208/2383] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:28.982102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd63e7a-cd6d-4803-b9e1-a2b40dcfc1c4 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:28.982371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd63e7a-cd6d-4803-b9e1-a2b40dcfc1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:28.982636 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd63e7a-cd6d-4803-b9e1-a2b40dcfc1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.005278 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-bfd63e7a-cd6d-4803-b9e1-a2b40dcfc1c4 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:29.006635 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-bfd63e7a-cd6d-4803-b9e1-a2b40dcfc1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:29.015873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1176/2384] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:29.020651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6d7c1-7920-4c8a-93b2-bbfca599f34f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.031562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6d7c1-7920-4c8a-93b2-bbfca599f34f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:29.037433 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4987e-8484-46cd-a1dc-d55ce56dc406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:29.038051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4987e-8484-46cd-a1dc-d55ce56dc406 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.038327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4987e-8484-46cd-a1dc-d55ce56dc406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.038600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4987e-8484-46cd-a1dc-d55ce56dc406 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.039251 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4987e-8484-46cd-a1dc-d55ce56dc406 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:42:29.039619 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4987e-8484-46cd-a1dc-d55ce56dc406 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:29.041238 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4987e-8484-46cd-a1dc-d55ce56dc406 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.066425 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1209/2385] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:42:29.071428 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd63e7a-cd6d-4803-b9e1-a2b40dcfc1c4 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.083657 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd63e7a-cd6d-4803-b9e1-a2b40dcfc1c4 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:29.088420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801b09f-0ea0-4fd2-9bee-2aa85568afc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:29.089034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801b09f-0ea0-4fd2-9bee-2aa85568afc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.089313 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801b09f-0ea0-4fd2-9bee-2aa85568afc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.089734 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801b09f-0ea0-4fd2-9bee-2aa85568afc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.090381 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801b09f-0ea0-4fd2-9bee-2aa85568afc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:29.092465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801b09f-0ea0-4fd2-9bee-2aa85568afc8 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.125699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1177/2386] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:29 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:29.132343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4987e-8484-46cd-a1dc-d55ce56dc406 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.141800 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4987e-8484-46cd-a1dc-d55ce56dc406 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:29.216587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc15f5-fee9-4344-af59-9d5481efe993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:29.217215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc15f5-fee9-4344-af59-9d5481efe993 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.217494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc15f5-fee9-4344-af59-9d5481efe993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.217765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc15f5-fee9-4344-af59-9d5481efe993 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.218415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc15f5-fee9-4344-af59-9d5481efe993 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:29.220532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc15f5-fee9-4344-af59-9d5481efe993 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.252612 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1210/2387] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:29 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:42:29.256791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801b09f-0ea0-4fd2-9bee-2aa85568afc8 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.270549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801b09f-0ea0-4fd2-9bee-2aa85568afc8 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:29.278680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd59c10-5ea5-4440-ab7d-11b8b2404f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:29.279617 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd59c10-5ea5-4440-ab7d-11b8b2404f1f admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.280086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd59c10-5ea5-4440-ab7d-11b8b2404f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.280529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd59c10-5ea5-4440-ab7d-11b8b2404f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.281355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd59c10-5ea5-4440-ab7d-11b8b2404f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:29.283165 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd59c10-5ea5-4440-ab7d-11b8b2404f1f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.294857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1178/2388] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:29.298858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc15f5-fee9-4344-af59-9d5481efe993 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.315672 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc15f5-fee9-4344-af59-9d5481efe993 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:29.321074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b9ca2-c7f4-4b34-a585-98db030ef61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:29.322184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b9ca2-c7f4-4b34-a585-98db030ef61d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.323009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b9ca2-c7f4-4b34-a585-98db030ef61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.323549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b9ca2-c7f4-4b34-a585-98db030ef61d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.324763 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b9ca2-c7f4-4b34-a585-98db030ef61d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:29.326336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b9ca2-c7f4-4b34-a585-98db030ef61d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.341145 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1211/2389] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:29.345528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd59c10-5ea5-4440-ab7d-11b8b2404f1f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.356652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd59c10-5ea5-4440-ab7d-11b8b2404f1f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:29.362432 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fc355-c4eb-42b1-975d-b2f92e5d930c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:29.363119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fc355-c4eb-42b1-975d-b2f92e5d930c admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.363394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fc355-c4eb-42b1-975d-b2f92e5d930c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.363664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fc355-c4eb-42b1-975d-b2f92e5d930c admin admin] PATH_INFO: `/v3/projects/443dfc2f02d245049712e6580775db58/users/018f1c33b76d494d8879a62b4afb4d65/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.380296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fc355-c4eb-42b1-975d-b2f92e5d930c admin admin] RBAC: Authorizing `identity:create_grant(project_id=443dfc2f02d245049712e6580775db58, user_id=018f1c33b76d494d8879a62b4afb4d65, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:29.383082 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fc355-c4eb-42b1-975d-b2f92e5d930c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.400778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1179/2390] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:29 2022] PUT /identity/v3/projects/443dfc2f02d245049712e6580775db58/users/018f1c33b76d494d8879a62b4afb4d65/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:29.411249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28e26-d520-47e0-9b84-50b1deb26de4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.411538 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28e26-d520-47e0-9b84-50b1deb26de4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.411983 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28e26-d520-47e0-9b84-50b1deb26de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.442750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-59d28e26-d520-47e0-9b84-50b1deb26de4 None None] MFA Rules not processed for user `018f1c33b76d494d8879a62b4afb4d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:29.444285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-59d28e26-d520-47e0-9b84-50b1deb26de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:29.527929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1212/2391] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:29.539621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fc355-c4eb-42b1-975d-b2f92e5d930c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:29.541662 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c48fc355-c4eb-42b1-975d-b2f92e5d930c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:29.568321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:29.616407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fc355-c4eb-42b1-975d-b2f92e5d930c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:29.620493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055eda3-8b28-4223-972f-a4378600ea95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:29.621315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055eda3-8b28-4223-972f-a4378600ea95 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:29.621594 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055eda3-8b28-4223-972f-a4378600ea95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:29.622892 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055eda3-8b28-4223-972f-a4378600ea95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:29.633185 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055eda3-8b28-4223-972f-a4378600ea95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:29.634668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055eda3-8b28-4223-972f-a4378600ea95 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:29.646719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1180/2392] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:39.034549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2169e-7a19-4eff-9624-e29d593e743e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:39.035061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2169e-7a19-4eff-9624-e29d593e743e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:39.035700 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2169e-7a19-4eff-9624-e29d593e743e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:39.058473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1fb2169e-7a19-4eff-9624-e29d593e743e None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:39.060011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1fb2169e-7a19-4eff-9624-e29d593e743e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:39.077487 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:39.139203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1213/2393] 10.222.0.62 () {58 vars in 967 bytes} [Thu Sep 1 16:42:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:39.151908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055eda3-8b28-4223-972f-a4378600ea95 service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:39.153540 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c055eda3-8b28-4223-972f-a4378600ea95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:39.245066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055eda3-8b28-4223-972f-a4378600ea95 service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:39.251267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b64d40c-936f-47dc-a11c-ce0be8e99e01 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:39.251969 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64d40c-936f-47dc-a11c-ce0be8e99e01 service swift] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:39.252224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64d40c-936f-47dc-a11c-ce0be8e99e01 service swift] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:39.252457 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64d40c-936f-47dc-a11c-ce0be8e99e01 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:39.263751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64d40c-936f-47dc-a11c-ce0be8e99e01 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:39.265554 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64d40c-936f-47dc-a11c-ce0be8e99e01 service swift] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:39.276034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1181/2394] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:42:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:42:41.900473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2169e-7a19-4eff-9624-e29d593e743e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:41.904333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1fb2169e-7a19-4eff-9624-e29d593e743e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:41.998220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2169e-7a19-4eff-9624-e29d593e743e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:42.004426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d1b90-5637-4c96-92f7-22e925b89dfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:42.005115 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d1b90-5637-4c96-92f7-22e925b89dfd service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.005610 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d1b90-5637-4c96-92f7-22e925b89dfd service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.006174 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d1b90-5637-4c96-92f7-22e925b89dfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.010032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7c5d1b90-5637-4c96-92f7-22e925b89dfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:42.025146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d1b90-5637-4c96-92f7-22e925b89dfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:42.026828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d1b90-5637-4c96-92f7-22e925b89dfd service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:42.042738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1214/2395] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:48.731925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b64d40c-936f-47dc-a11c-ce0be8e99e01 service swift] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:48.821556 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b64d40c-936f-47dc-a11c-ce0be8e99e01 service swift] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:48.827424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff266ed-d7fa-43f4-9a8b-3a4a7ebff5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:48.828411 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff266ed-d7fa-43f4-9a8b-3a4a7ebff5d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:48.829018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff266ed-d7fa-43f4-9a8b-3a4a7ebff5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:48.829348 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff266ed-d7fa-43f4-9a8b-3a4a7ebff5d1 admin admin] PATH_INFO: `/v3/users/018f1c33b76d494d8879a62b4afb4d65` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:48.830619 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff266ed-d7fa-43f4-9a8b-3a4a7ebff5d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=018f1c33b76d494d8879a62b4afb4d65)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:48.832717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff266ed-d7fa-43f4-9a8b-3a4a7ebff5d1 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:48.941005 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cff266ed-d7fa-43f4-9a8b-3a4a7ebff5d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018f1c33b76d494d8879a62b4afb4d65'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:48.950899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:48.972598 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cff266ed-d7fa-43f4-9a8b-3a4a7ebff5d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018f1c33b76d494d8879a62b4afb4d65'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:48.984963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cff266ed-d7fa-43f4-9a8b-3a4a7ebff5d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018f1c33b76d494d8879a62b4afb4d65'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.000350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cff266ed-d7fa-43f4-9a8b-3a4a7ebff5d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018f1c33b76d494d8879a62b4afb4d65'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.001184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cff266ed-d7fa-43f4-9a8b-3a4a7ebff5d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018f1c33b76d494d8879a62b4afb4d65'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.009424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1182/2396] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:48 2022] DELETE /identity/v3/users/018f1c33b76d494d8879a62b4afb4d65 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:49.014848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4109b-b3be-470a-b2e0-fd5d7c41dfbe None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:49.015379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4109b-b3be-470a-b2e0-fd5d7c41dfbe None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:49.015472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4109b-b3be-470a-b2e0-fd5d7c41dfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:49.049984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b6f4109b-b3be-470a-b2e0-fd5d7c41dfbe None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:49.049984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b6f4109b-b3be-470a-b2e0-fd5d7c41dfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:49.112087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1215/2397] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:42:49.123571 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff266ed-d7fa-43f4-9a8b-3a4a7ebff5d1 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:49.125258 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-cff266ed-d7fa-43f4-9a8b-3a4a7ebff5d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:49.185088 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff266ed-d7fa-43f4-9a8b-3a4a7ebff5d1 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:49.189373 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7114f-078b-4182-9afb-653412d0362b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:49.189987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7114f-078b-4182-9afb-653412d0362b service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:49.190262 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7114f-078b-4182-9afb-653412d0362b service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:49.190528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7114f-078b-4182-9afb-653412d0362b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:49.201181 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7114f-078b-4182-9afb-653412d0362b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:49.202615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7114f-078b-4182-9afb-653412d0362b service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:49.212240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1183/2398] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:49.498411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4109b-b3be-470a-b2e0-fd5d7c41dfbe None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:49.499971 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b6f4109b-b3be-470a-b2e0-fd5d7c41dfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:49.517488 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4109b-b3be-470a-b2e0-fd5d7c41dfbe None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:49.523449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:49.524104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:49.524406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:49.524663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] PATH_INFO: `/v3/projects/443dfc2f02d245049712e6580775db58` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:49.525812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=443dfc2f02d245049712e6580775db58)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:49.527873 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:49.591662 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:49.597555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '443dfc2f02d245049712e6580775db58', 'request_id': 'req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.606290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '443dfc2f02d245049712e6580775db58', 'request_id': 'req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.615003 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '443dfc2f02d245049712e6580775db58', 'request_id': 'req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.630216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] The token cache is being invalidate because project 443dfc2f02d245049712e6580775db58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:49.630453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.631134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.633040 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1216/2399] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:49 2022] DELETE /identity/v3/projects/443dfc2f02d245049712e6580775db58 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:50.207083 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7114f-078b-4182-9afb-653412d0362b service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:50.209140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-5af7114f-078b-4182-9afb-653412d0362b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:50.221294 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7114f-078b-4182-9afb-653412d0362b service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:50.278382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d772d2-9f1c-4bdc-9c25-209635c380e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:50.278382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d772d2-9f1c-4bdc-9c25-209635c380e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:50.278382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d772d2-9f1c-4bdc-9c25-209635c380e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:50.278382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d772d2-9f1c-4bdc-9c25-209635c380e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:50.279464 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d0d772d2-9f1c-4bdc-9c25-209635c380e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:50.351006 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d772d2-9f1c-4bdc-9c25-209635c380e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:50.352529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d772d2-9f1c-4bdc-9c25-209635c380e1 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:50.363639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1184/2400] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:55.276679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:55.278142 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:55.293117 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f8f2c-92a6-4a03-95df-2c18cecabc0c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:55.298819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c9032-9334-4203-8cd5-8e13894858f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:55.299463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c9032-9334-4203-8cd5-8e13894858f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.299731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c9032-9334-4203-8cd5-8e13894858f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.300025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c9032-9334-4203-8cd5-8e13894858f5 admin admin] PATH_INFO: `/v3/users/2c1c3f8f28b147fc838533baf62b72af` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.302157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c9032-9334-4203-8cd5-8e13894858f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c1c3f8f28b147fc838533baf62b72af)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:55.304123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c9032-9334-4203-8cd5-8e13894858f5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:55.364580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-166c9032-9334-4203-8cd5-8e13894858f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c1c3f8f28b147fc838533baf62b72af'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:55.373990 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-166c9032-9334-4203-8cd5-8e13894858f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c1c3f8f28b147fc838533baf62b72af'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:55.374602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-166c9032-9334-4203-8cd5-8e13894858f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c1c3f8f28b147fc838533baf62b72af'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:55.388297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-166c9032-9334-4203-8cd5-8e13894858f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c1c3f8f28b147fc838533baf62b72af'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:55.395445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-166c9032-9334-4203-8cd5-8e13894858f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c1c3f8f28b147fc838533baf62b72af'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:55.402670 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1217/2401] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:55 2022] DELETE /identity/v3/users/2c1c3f8f28b147fc838533baf62b72af => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:55.587353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d772d2-9f1c-4bdc-9c25-209635c380e1 service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:55.600359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d772d2-9f1c-4bdc-9c25-209635c380e1 service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:55.625382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:55.666732 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b4789-e6ce-428a-a9ac-ee5333babddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:55.667280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b4789-e6ce-428a-a9ac-ee5333babddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.667558 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b4789-e6ce-428a-a9ac-ee5333babddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.667831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b4789-e6ce-428a-a9ac-ee5333babddf admin admin] PATH_INFO: `/v3/projects/af2ea34aea99498eb36647be71aec5e8` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.669107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b4789-e6ce-428a-a9ac-ee5333babddf admin admin] RBAC: Authorizing `identity:delete_project(project_id=af2ea34aea99498eb36647be71aec5e8)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:55.671090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b4789-e6ce-428a-a9ac-ee5333babddf admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:55.721259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bc4b4789-e6ce-428a-a9ac-ee5333babddf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af2ea34aea99498eb36647be71aec5e8', 'request_id': 'req-bc4b4789-e6ce-428a-a9ac-ee5333babddf'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:55.729669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bc4b4789-e6ce-428a-a9ac-ee5333babddf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af2ea34aea99498eb36647be71aec5e8', 'request_id': 'req-bc4b4789-e6ce-428a-a9ac-ee5333babddf'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:55.735568 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bc4b4789-e6ce-428a-a9ac-ee5333babddf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af2ea34aea99498eb36647be71aec5e8', 'request_id': 'req-bc4b4789-e6ce-428a-a9ac-ee5333babddf'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:55.747462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bc4b4789-e6ce-428a-a9ac-ee5333babddf admin admin] The token cache is being invalidate because project af2ea34aea99498eb36647be71aec5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:55.747751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bc4b4789-e6ce-428a-a9ac-ee5333babddf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:55.748553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bc4b4789-e6ce-428a-a9ac-ee5333babddf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:55.750371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1185/2402] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:55 2022] DELETE /identity/v3/projects/af2ea34aea99498eb36647be71aec5e8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:55.755288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278502f6-c468-4512-8911-c9bfe23886a2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.755661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278502f6-c468-4512-8911-c9bfe23886a2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.755851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278502f6-c468-4512-8911-c9bfe23886a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.777507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-278502f6-c468-4512-8911-c9bfe23886a2 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:55.778797 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-278502f6-c468-4512-8911-c9bfe23886a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:55.852174 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1218/2403] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:55.860575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75429a-4138-4d07-8bd0-4ac114f32251 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.860931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75429a-4138-4d07-8bd0-4ac114f32251 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.861289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75429a-4138-4d07-8bd0-4ac114f32251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.881949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-bc75429a-4138-4d07-8bd0-4ac114f32251 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:55.883146 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-bc75429a-4138-4d07-8bd0-4ac114f32251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:55.895073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1186/2404] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:55.899850 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278502f6-c468-4512-8911-c9bfe23886a2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:55.909985 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278502f6-c468-4512-8911-c9bfe23886a2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:55.914775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ddc2f-46c5-40b1-9ff2-1f59e1ff57e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:55.915379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ddc2f-46c5-40b1-9ff2-1f59e1ff57e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.915657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ddc2f-46c5-40b1-9ff2-1f59e1ff57e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.915988 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ddc2f-46c5-40b1-9ff2-1f59e1ff57e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.916961 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ddc2f-46c5-40b1-9ff2-1f59e1ff57e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:42:55.917222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ddc2f-46c5-40b1-9ff2-1f59e1ff57e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:55.918897 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ddc2f-46c5-40b1-9ff2-1f59e1ff57e4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:55.933765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1219/2405] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:42:55.938657 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc75429a-4138-4d07-8bd0-4ac114f32251 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:55.949852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc75429a-4138-4d07-8bd0-4ac114f32251 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:55.960257 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b766af8-2682-4869-842c-0995700614ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:55.960860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b766af8-2682-4869-842c-0995700614ce admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:55.961136 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b766af8-2682-4869-842c-0995700614ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:55.961403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b766af8-2682-4869-842c-0995700614ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.962102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b766af8-2682-4869-842c-0995700614ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:55.964140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b766af8-2682-4869-842c-0995700614ce admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:55.997471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1187/2406] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:55 2022] POST /identity/v3/projects => generated 361 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:56.002522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ddc2f-46c5-40b1-9ff2-1f59e1ff57e4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:56.014447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ddc2f-46c5-40b1-9ff2-1f59e1ff57e4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:56.050428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:56.081827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fa2f3-a910-48cd-bbfa-9eb1c3a79ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:56.082731 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fa2f3-a910-48cd-bbfa-9eb1c3a79ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:56.083181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fa2f3-a910-48cd-bbfa-9eb1c3a79ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:56.083496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fa2f3-a910-48cd-bbfa-9eb1c3a79ef1 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:56.084322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fa2f3-a910-48cd-bbfa-9eb1c3a79ef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:56.086664 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fa2f3-a910-48cd-bbfa-9eb1c3a79ef1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:56.110873 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1220/2407] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:56 2022] POST /identity/v3/users => generated 281 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:56.117241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b766af8-2682-4869-842c-0995700614ce admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:56.133469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b766af8-2682-4869-842c-0995700614ce admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:56.139801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c33dfe-9414-45ae-b320-d9b67618405f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:56.140581 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c33dfe-9414-45ae-b320-d9b67618405f admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:56.140860 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c33dfe-9414-45ae-b320-d9b67618405f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:56.141185 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c33dfe-9414-45ae-b320-d9b67618405f admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:56.141906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c33dfe-9414-45ae-b320-d9b67618405f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:56.143911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c33dfe-9414-45ae-b320-d9b67618405f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:56.156834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1188/2408] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:56.162276 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fa2f3-a910-48cd-bbfa-9eb1c3a79ef1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:56.175576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fa2f3-a910-48cd-bbfa-9eb1c3a79ef1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:56.181011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a5986-4a1f-434f-845f-6336a24b72f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:56.181632 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a5986-4a1f-434f-845f-6336a24b72f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:56.182031 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a5986-4a1f-434f-845f-6336a24b72f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:56.182304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a5986-4a1f-434f-845f-6336a24b72f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:56.183134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a5986-4a1f-434f-845f-6336a24b72f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:56.184763 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a5986-4a1f-434f-845f-6336a24b72f7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:56.196185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1221/2409] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:56.201759 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c33dfe-9414-45ae-b320-d9b67618405f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:56.212716 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c33dfe-9414-45ae-b320-d9b67618405f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:56.218855 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a8131-b817-468e-a105-4620a9b7d8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:56.219529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a8131-b817-468e-a105-4620a9b7d8be admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:56.219813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a8131-b817-468e-a105-4620a9b7d8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:56.220114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a8131-b817-468e-a105-4620a9b7d8be admin admin] PATH_INFO: `/v3/projects/da1df3c6980a495fa5344417e8d8cdd9/users/3e56a6388bbf4d10bac28c21c56273ea/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:56.236767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a8131-b817-468e-a105-4620a9b7d8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=da1df3c6980a495fa5344417e8d8cdd9, user_id=3e56a6388bbf4d10bac28c21c56273ea, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:56.239522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a8131-b817-468e-a105-4620a9b7d8be admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:56.253851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1189/2410] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:56 2022] PUT /identity/v3/projects/da1df3c6980a495fa5344417e8d8cdd9/users/3e56a6388bbf4d10bac28c21c56273ea/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:56.265899 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db702d-23ef-4d68-ae5d-6e36c0027b32 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:56.266149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db702d-23ef-4d68-ae5d-6e36c0027b32 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:56.266441 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db702d-23ef-4d68-ae5d-6e36c0027b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:56.300168 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-72db702d-23ef-4d68-ae5d-6e36c0027b32 None None] MFA Rules not processed for user `3e56a6388bbf4d10bac28c21c56273ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:56.301780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-72db702d-23ef-4d68-ae5d-6e36c0027b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:56.373647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1222/2411] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:56.385449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a8131-b817-468e-a105-4620a9b7d8be admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:56.468179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:56.484528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a8131-b817-468e-a105-4620a9b7d8be admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:56.491389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53a157-9759-4c5d-80ec-38d583347d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:56.492135 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53a157-9759-4c5d-80ec-38d583347d56 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:56.492479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53a157-9759-4c5d-80ec-38d583347d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:56.492817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53a157-9759-4c5d-80ec-38d583347d56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:56.493696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53a157-9759-4c5d-80ec-38d583347d56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:56.495936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53a157-9759-4c5d-80ec-38d583347d56 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:56.529014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1190/2412] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:56 2022] POST /identity/v3/projects => generated 363 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:56.530266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db702d-23ef-4d68-ae5d-6e36c0027b32 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:56.541894 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db702d-23ef-4d68-ae5d-6e36c0027b32 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:56.603718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2430b9-018d-4577-8e7b-176942355ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:56.604413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2430b9-018d-4577-8e7b-176942355ffe admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:56.604685 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2430b9-018d-4577-8e7b-176942355ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:56.604953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2430b9-018d-4577-8e7b-176942355ffe admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:56.605580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2430b9-018d-4577-8e7b-176942355ffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:56.607498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2430b9-018d-4577-8e7b-176942355ffe admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:56.631258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1223/2413] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:56 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:42:56.635524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53a157-9759-4c5d-80ec-38d583347d56 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:56.645990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53a157-9759-4c5d-80ec-38d583347d56 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:56.651753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3adac6-540b-4a46-a68b-89fead745999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:56.652385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3adac6-540b-4a46-a68b-89fead745999 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:56.652659 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3adac6-540b-4a46-a68b-89fead745999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:56.652925 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3adac6-540b-4a46-a68b-89fead745999 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:56.653559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3adac6-540b-4a46-a68b-89fead745999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:56.655162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3adac6-540b-4a46-a68b-89fead745999 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:56.667032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1191/2414] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:56.673114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2430b9-018d-4577-8e7b-176942355ffe admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:56.684848 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2430b9-018d-4577-8e7b-176942355ffe admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:56.690337 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af8852-98f0-4e22-8c55-4f0269a4f7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:56.691038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af8852-98f0-4e22-8c55-4f0269a4f7b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:56.691314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af8852-98f0-4e22-8c55-4f0269a4f7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:56.691579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af8852-98f0-4e22-8c55-4f0269a4f7b0 admin admin] PATH_INFO: `/v3/projects/f1b35790f9dd4e5d9dfb6301460bd172/users/9bf112f199194baeb2840d2cec9c7229/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:56.708371 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af8852-98f0-4e22-8c55-4f0269a4f7b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b35790f9dd4e5d9dfb6301460bd172, user_id=9bf112f199194baeb2840d2cec9c7229, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:56.711017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af8852-98f0-4e22-8c55-4f0269a4f7b0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:56.730192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1224/2415] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:56 2022] PUT /identity/v3/projects/f1b35790f9dd4e5d9dfb6301460bd172/users/9bf112f199194baeb2840d2cec9c7229/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:56.735456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3adac6-540b-4a46-a68b-89fead745999 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:56.746323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3adac6-540b-4a46-a68b-89fead745999 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:56.805785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132194b-a3b1-4374-b324-2330778d9358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:56.806399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132194b-a3b1-4374-b324-2330778d9358 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:56.806668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132194b-a3b1-4374-b324-2330778d9358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:56.807011 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132194b-a3b1-4374-b324-2330778d9358 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:56.807645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132194b-a3b1-4374-b324-2330778d9358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:56.812804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132194b-a3b1-4374-b324-2330778d9358 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:56.824982 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1192/2416] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:56.829436 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af8852-98f0-4e22-8c55-4f0269a4f7b0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:56.840609 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:56.844516 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af8852-98f0-4e22-8c55-4f0269a4f7b0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:56.851749 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1aafa-af73-4419-85e1-05f30d7cd431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:56.852450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1aafa-af73-4419-85e1-05f30d7cd431 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:56.852765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1aafa-af73-4419-85e1-05f30d7cd431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:56.853034 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1aafa-af73-4419-85e1-05f30d7cd431 admin admin] PATH_INFO: `/v3/projects/f1b35790f9dd4e5d9dfb6301460bd172/users/9bf112f199194baeb2840d2cec9c7229/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:56.855459 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1aafa-af73-4419-85e1-05f30d7cd431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b35790f9dd4e5d9dfb6301460bd172, user_id=9bf112f199194baeb2840d2cec9c7229, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:56.858358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1aafa-af73-4419-85e1-05f30d7cd431 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:56.872437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1225/2417] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:56 2022] PUT /identity/v3/projects/f1b35790f9dd4e5d9dfb6301460bd172/users/9bf112f199194baeb2840d2cec9c7229/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:56.884328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fdef2-7864-4f91-af92-c97d54f67793 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:56.884615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fdef2-7864-4f91-af92-c97d54f67793 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:56.884887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fdef2-7864-4f91-af92-c97d54f67793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:56.914016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-c34fdef2-7864-4f91-af92-c97d54f67793 None None] MFA Rules not processed for user `9bf112f199194baeb2840d2cec9c7229`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:56.915228 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c34fdef2-7864-4f91-af92-c97d54f67793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:56.988116 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1193/2418] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:57.000164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1aafa-af73-4419-85e1-05f30d7cd431 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:57.002807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b1c1aafa-af73-4419-85e1-05f30d7cd431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:57.075825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1aafa-af73-4419-85e1-05f30d7cd431 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:57.079817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8e06b-a6e4-4a0f-a7c0-cc71c5ab28b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:57.080476 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8e06b-a6e4-4a0f-a7c0-cc71c5ab28b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:57.080877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8e06b-a6e4-4a0f-a7c0-cc71c5ab28b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:57.081215 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8e06b-a6e4-4a0f-a7c0-cc71c5ab28b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:57.091435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8e06b-a6e4-4a0f-a7c0-cc71c5ab28b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:57.092911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8e06b-a6e4-4a0f-a7c0-cc71c5ab28b1 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:57.107261 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1226/2419] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:57.212307 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fdef2-7864-4f91-af92-c97d54f67793 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:57.224819 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fdef2-7864-4f91-af92-c97d54f67793 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:57.228969 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba8d17-6437-4e76-9b7f-e3d853a76b4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:57.229510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba8d17-6437-4e76-9b7f-e3d853a76b4e service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:57.229804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba8d17-6437-4e76-9b7f-e3d853a76b4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:57.230186 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba8d17-6437-4e76-9b7f-e3d853a76b4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:57.285824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:57.304929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba8d17-6437-4e76-9b7f-e3d853a76b4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:57.307273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba8d17-6437-4e76-9b7f-e3d853a76b4e service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:57.318198 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1194/2420] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:58.439637 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8e06b-a6e4-4a0f-a7c0-cc71c5ab28b1 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:58.441271 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f8e8e06b-a6e4-4a0f-a7c0-cc71c5ab28b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:58.525409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8e06b-a6e4-4a0f-a7c0-cc71c5ab28b1 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:58.529374 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29cfa0-8a9f-41e6-a1e7-883aac4be109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:58.529988 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29cfa0-8a9f-41e6-a1e7-883aac4be109 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:58.530258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29cfa0-8a9f-41e6-a1e7-883aac4be109 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:58.530526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29cfa0-8a9f-41e6-a1e7-883aac4be109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:58.532296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-af29cfa0-8a9f-41e6-a1e7-883aac4be109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:58.622296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29cfa0-8a9f-41e6-a1e7-883aac4be109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:58.623771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29cfa0-8a9f-41e6-a1e7-883aac4be109 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:58.634942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1227/2421] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:02.203227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba8d17-6437-4e76-9b7f-e3d853a76b4e service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:02.204724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e1ba8d17-6437-4e76-9b7f-e3d853a76b4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:02.220837 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba8d17-6437-4e76-9b7f-e3d853a76b4e service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:02.225806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3ed0f-1da6-486a-9330-dfb7400e0469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:02.226449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3ed0f-1da6-486a-9330-dfb7400e0469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:02.226722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3ed0f-1da6-486a-9330-dfb7400e0469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:02.226981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3ed0f-1da6-486a-9330-dfb7400e0469 admin admin] PATH_INFO: `/v3/users/3a9b222396b1438f9fa0af0e197a9293` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:02.228277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3ed0f-1da6-486a-9330-dfb7400e0469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a9b222396b1438f9fa0af0e197a9293)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:02.230596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3ed0f-1da6-486a-9330-dfb7400e0469 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:02.295655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2dd3ed0f-1da6-486a-9330-dfb7400e0469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a9b222396b1438f9fa0af0e197a9293'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:02.311400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2dd3ed0f-1da6-486a-9330-dfb7400e0469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a9b222396b1438f9fa0af0e197a9293'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:02.319494 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2dd3ed0f-1da6-486a-9330-dfb7400e0469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a9b222396b1438f9fa0af0e197a9293'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:02.330005 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2dd3ed0f-1da6-486a-9330-dfb7400e0469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a9b222396b1438f9fa0af0e197a9293'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:02.330637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2dd3ed0f-1da6-486a-9330-dfb7400e0469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a9b222396b1438f9fa0af0e197a9293'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:02.339188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1195/2422] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:02 2022] DELETE /identity/v3/users/3a9b222396b1438f9fa0af0e197a9293 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:02.576106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29cfa0-8a9f-41e6-a1e7-883aac4be109 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:02.588046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29cfa0-8a9f-41e6-a1e7-883aac4be109 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:02.668493 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:02.692432 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c5353-e7c0-4949-881a-d91fa776483a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:02.693716 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c5353-e7c0-4949-881a-d91fa776483a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:02.694260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c5353-e7c0-4949-881a-d91fa776483a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:02.694782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c5353-e7c0-4949-881a-d91fa776483a admin admin] PATH_INFO: `/v3/projects/c6eb3ba21eae45898f42f3456591591e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:02.697898 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c5353-e7c0-4949-881a-d91fa776483a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6eb3ba21eae45898f42f3456591591e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:02.702206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c5353-e7c0-4949-881a-d91fa776483a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:02.790338 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2a2c5353-e7c0-4949-881a-d91fa776483a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6eb3ba21eae45898f42f3456591591e', 'request_id': 'req-2a2c5353-e7c0-4949-881a-d91fa776483a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:02.800953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2a2c5353-e7c0-4949-881a-d91fa776483a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6eb3ba21eae45898f42f3456591591e', 'request_id': 'req-2a2c5353-e7c0-4949-881a-d91fa776483a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:02.810787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2a2c5353-e7c0-4949-881a-d91fa776483a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6eb3ba21eae45898f42f3456591591e', 'request_id': 'req-2a2c5353-e7c0-4949-881a-d91fa776483a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:02.830236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2a2c5353-e7c0-4949-881a-d91fa776483a admin admin] The token cache is being invalidate because project c6eb3ba21eae45898f42f3456591591e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:02.830568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2a2c5353-e7c0-4949-881a-d91fa776483a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:02.831804 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2a2c5353-e7c0-4949-881a-d91fa776483a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:02.834588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1228/2423] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:02 2022] DELETE /identity/v3/projects/c6eb3ba21eae45898f42f3456591591e => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:02.841391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d92f0c-cab3-47a1-8f8f-874c540ac063 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:02.841796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d92f0c-cab3-47a1-8f8f-874c540ac063 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:02.842119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d92f0c-cab3-47a1-8f8f-874c540ac063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:02.870149 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-f8d92f0c-cab3-47a1-8f8f-874c540ac063 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:02.871668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f8d92f0c-cab3-47a1-8f8f-874c540ac063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:02.965207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1196/2424] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:02.975921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768295d1-f615-4079-8b09-8e4ef90a25a6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:02.976652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768295d1-f615-4079-8b09-8e4ef90a25a6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:02.977345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768295d1-f615-4079-8b09-8e4ef90a25a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:03.010408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-768295d1-f615-4079-8b09-8e4ef90a25a6 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:03.012670 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-768295d1-f615-4079-8b09-8e4ef90a25a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:03.026084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1229/2425] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:03.031498 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d92f0c-cab3-47a1-8f8f-874c540ac063 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:03.043387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d92f0c-cab3-47a1-8f8f-874c540ac063 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:03.049426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b65064-9164-4c71-897b-41d54d9df43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:03.050028 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b65064-9164-4c71-897b-41d54d9df43d admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:03.050301 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b65064-9164-4c71-897b-41d54d9df43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:03.050559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b65064-9164-4c71-897b-41d54d9df43d admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:03.051210 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b65064-9164-4c71-897b-41d54d9df43d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:03.051502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b65064-9164-4c71-897b-41d54d9df43d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:03.053125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b65064-9164-4c71-897b-41d54d9df43d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:03.069924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1197/2426] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:03.074939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768295d1-f615-4079-8b09-8e4ef90a25a6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:03.084631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768295d1-f615-4079-8b09-8e4ef90a25a6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:03.090149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffaeac9-1186-4ceb-9a69-4ce4c824a27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:03.090744 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffaeac9-1186-4ceb-9a69-4ce4c824a27b admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:03.091137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffaeac9-1186-4ceb-9a69-4ce4c824a27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:03.091282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffaeac9-1186-4ceb-9a69-4ce4c824a27b admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:03.091959 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffaeac9-1186-4ceb-9a69-4ce4c824a27b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:03.095447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffaeac9-1186-4ceb-9a69-4ce4c824a27b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:03.131128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1230/2427] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:03 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:03.133617 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b65064-9164-4c71-897b-41d54d9df43d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:03.144813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b65064-9164-4c71-897b-41d54d9df43d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:03.188599 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:03.212608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee25275-3db7-4aac-8209-6339711edf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:03.213618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee25275-3db7-4aac-8209-6339711edf34 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:03.213904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee25275-3db7-4aac-8209-6339711edf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:03.214174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee25275-3db7-4aac-8209-6339711edf34 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:03.214818 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee25275-3db7-4aac-8209-6339711edf34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:03.216786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee25275-3db7-4aac-8209-6339711edf34 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:03.245186 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1198/2428] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:03 2022] POST /identity/v3/users => generated 303 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:03.246526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffaeac9-1186-4ceb-9a69-4ce4c824a27b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:03.258236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffaeac9-1186-4ceb-9a69-4ce4c824a27b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:03.263855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4f5d5-6842-4a76-91f7-5ae7bc49791f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:03.264537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4f5d5-6842-4a76-91f7-5ae7bc49791f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:03.264846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4f5d5-6842-4a76-91f7-5ae7bc49791f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:03.265151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4f5d5-6842-4a76-91f7-5ae7bc49791f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:03.265856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4f5d5-6842-4a76-91f7-5ae7bc49791f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:03.267499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4f5d5-6842-4a76-91f7-5ae7bc49791f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:03.279426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1231/2429] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:03.283753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee25275-3db7-4aac-8209-6339711edf34 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:03.294760 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee25275-3db7-4aac-8209-6339711edf34 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:03.299984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ad00a-cd06-4ad1-874e-5df6d37dbfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:03.300627 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ad00a-cd06-4ad1-874e-5df6d37dbfff admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:03.300994 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ad00a-cd06-4ad1-874e-5df6d37dbfff admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:03.301266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ad00a-cd06-4ad1-874e-5df6d37dbfff admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:03.301937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ad00a-cd06-4ad1-874e-5df6d37dbfff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:03.303592 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ad00a-cd06-4ad1-874e-5df6d37dbfff admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:03.315275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1199/2430] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:03.319990 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4f5d5-6842-4a76-91f7-5ae7bc49791f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:03.331372 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4f5d5-6842-4a76-91f7-5ae7bc49791f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:03.336307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01d415-14ec-44a5-9ac9-4eb2fb51bc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:03.336993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01d415-14ec-44a5-9ac9-4eb2fb51bc0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:03.337277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01d415-14ec-44a5-9ac9-4eb2fb51bc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:03.337549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01d415-14ec-44a5-9ac9-4eb2fb51bc0e admin admin] PATH_INFO: `/v3/projects/1555e6f3a9f04390b4d021c5d64c5cfa/users/59cfcf9864444a76885768b3a4fd5842/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:03.356114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01d415-14ec-44a5-9ac9-4eb2fb51bc0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1555e6f3a9f04390b4d021c5d64c5cfa, user_id=59cfcf9864444a76885768b3a4fd5842, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:03.358739 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01d415-14ec-44a5-9ac9-4eb2fb51bc0e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:03.378788 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1232/2431] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:03 2022] PUT /identity/v3/projects/1555e6f3a9f04390b4d021c5d64c5cfa/users/59cfcf9864444a76885768b3a4fd5842/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:03.384922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fed458-6199-45c1-b70a-a9979a73434c None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:03.385471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fed458-6199-45c1-b70a-a9979a73434c None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:03.385895 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fed458-6199-45c1-b70a-a9979a73434c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:03.407719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-41fed458-6199-45c1-b70a-a9979a73434c None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:03.408877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-41fed458-6199-45c1-b70a-a9979a73434c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:03.474714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1200/2432] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:03.484816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01d415-14ec-44a5-9ac9-4eb2fb51bc0e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:03.486626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dc01d415-14ec-44a5-9ac9-4eb2fb51bc0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:03.550076 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01d415-14ec-44a5-9ac9-4eb2fb51bc0e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:03.553928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b8b54-03e8-490b-a31e-2f0d0bfe34ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:03.554537 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b8b54-03e8-490b-a31e-2f0d0bfe34ab service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:03.554815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b8b54-03e8-490b-a31e-2f0d0bfe34ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:03.555073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b8b54-03e8-490b-a31e-2f0d0bfe34ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:03.565767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b8b54-03e8-490b-a31e-2f0d0bfe34ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:03.567366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b8b54-03e8-490b-a31e-2f0d0bfe34ab service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:03.578676 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1233/2433] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:04.418634 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938bbcc-6dd3-4831-a0aa-2511191f4e6d None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:04.420491 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938bbcc-6dd3-4831-a0aa-2511191f4e6d None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:04.421002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938bbcc-6dd3-4831-a0aa-2511191f4e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:04.466753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-1938bbcc-6dd3-4831-a0aa-2511191f4e6d None None] MFA Rules not processed for user `59cfcf9864444a76885768b3a4fd5842`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:04.468418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1938bbcc-6dd3-4831-a0aa-2511191f4e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:04.602895 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1201/2434] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:04 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:04.608686 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b8b54-03e8-490b-a31e-2f0d0bfe34ab service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:04.621112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b8b54-03e8-490b-a31e-2f0d0bfe34ab service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:04.643541 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8104a94-a6bc-41cf-bc30-ae0cf21c9144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:04.644300 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8104a94-a6bc-41cf-bc30-ae0cf21c9144 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:04.644585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8104a94-a6bc-41cf-bc30-ae0cf21c9144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:04.644858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8104a94-a6bc-41cf-bc30-ae0cf21c9144 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:04.646673 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8104a94-a6bc-41cf-bc30-ae0cf21c9144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:04.649322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8104a94-a6bc-41cf-bc30-ae0cf21c9144 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:04.664612 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:04.689204 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1234/2435] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:04 2022] POST /identity/v3/projects => generated 403 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:04.692618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938bbcc-6dd3-4831-a0aa-2511191f4e6d None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:04.710729 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938bbcc-6dd3-4831-a0aa-2511191f4e6d None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:04.791617 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:04.807868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261851a-06a2-4cf4-949b-a40ed3729076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:04.808867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261851a-06a2-4cf4-949b-a40ed3729076 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:04.809375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261851a-06a2-4cf4-949b-a40ed3729076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:04.809648 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261851a-06a2-4cf4-949b-a40ed3729076 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:04.810351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5261851a-06a2-4cf4-949b-a40ed3729076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:04.812648 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5261851a-06a2-4cf4-949b-a40ed3729076 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:04.843843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1202/2436] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:04 2022] POST /identity/v3/users => generated 308 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:04.844890 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8104a94-a6bc-41cf-bc30-ae0cf21c9144 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:04.855966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8104a94-a6bc-41cf-bc30-ae0cf21c9144 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:04.860951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a53f47-b89a-4d31-9c88-7491df143360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:04.861568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a53f47-b89a-4d31-9c88-7491df143360 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:04.861861 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a53f47-b89a-4d31-9c88-7491df143360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:04.862115 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a53f47-b89a-4d31-9c88-7491df143360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:04.862765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a53f47-b89a-4d31-9c88-7491df143360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:04.864469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a53f47-b89a-4d31-9c88-7491df143360 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:04.876615 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1235/2437] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:04.881586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261851a-06a2-4cf4-949b-a40ed3729076 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:04.893197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261851a-06a2-4cf4-949b-a40ed3729076 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:04.902434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3aecce-581d-4c06-b527-93b7df67b9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:04.903171 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aecce-581d-4c06-b527-93b7df67b9e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:04.903467 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aecce-581d-4c06-b527-93b7df67b9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:04.904814 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aecce-581d-4c06-b527-93b7df67b9e2 admin admin] PATH_INFO: `/v3/projects/c334c68a88084957a49432fd95d4e165/users/eca8248d524244e0b4ef0f4fa2ff3470/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:04.924939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3aecce-581d-4c06-b527-93b7df67b9e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c334c68a88084957a49432fd95d4e165, user_id=eca8248d524244e0b4ef0f4fa2ff3470, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:04.928398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3aecce-581d-4c06-b527-93b7df67b9e2 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:04.950877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1203/2438] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:04 2022] PUT /identity/v3/projects/c334c68a88084957a49432fd95d4e165/users/eca8248d524244e0b4ef0f4fa2ff3470/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:04.958440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a53f47-b89a-4d31-9c88-7491df143360 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:04.971678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a53f47-b89a-4d31-9c88-7491df143360 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:05.053490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db726b5-d544-454f-b349-bf3b24bc5983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:05.053995 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db726b5-d544-454f-b349-bf3b24bc5983 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:05.054338 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db726b5-d544-454f-b349-bf3b24bc5983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:05.054613 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db726b5-d544-454f-b349-bf3b24bc5983 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:05.055264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db726b5-d544-454f-b349-bf3b24bc5983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:05.057331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db726b5-d544-454f-b349-bf3b24bc5983 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:05.069344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1236/2439] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:05.074254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3aecce-581d-4c06-b527-93b7df67b9e2 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:05.086320 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3aecce-581d-4c06-b527-93b7df67b9e2 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:05.092641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a2f23-db14-4aa2-add0-e1e09f27ee67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:05.093505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a2f23-db14-4aa2-add0-e1e09f27ee67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:05.093867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a2f23-db14-4aa2-add0-e1e09f27ee67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:05.094233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a2f23-db14-4aa2-add0-e1e09f27ee67 admin admin] PATH_INFO: `/v3/projects/c334c68a88084957a49432fd95d4e165/users/eca8248d524244e0b4ef0f4fa2ff3470/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:05.096289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a2f23-db14-4aa2-add0-e1e09f27ee67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c334c68a88084957a49432fd95d4e165, user_id=eca8248d524244e0b4ef0f4fa2ff3470, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:05.099163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a2f23-db14-4aa2-add0-e1e09f27ee67 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:05.111599 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1204/2440] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:05 2022] PUT /identity/v3/projects/c334c68a88084957a49432fd95d4e165/users/eca8248d524244e0b4ef0f4fa2ff3470/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:05.988293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6592ab-bda3-4e8b-a6c8-06460eb5322b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:05.989061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6592ab-bda3-4e8b-a6c8-06460eb5322b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:05.989565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6592ab-bda3-4e8b-a6c8-06460eb5322b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:06.034934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ce6592ab-bda3-4e8b-a6c8-06460eb5322b None None] MFA Rules not processed for user `eca8248d524244e0b4ef0f4fa2ff3470`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:06.037772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ce6592ab-bda3-4e8b-a6c8-06460eb5322b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:06.166745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1237/2441] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:05 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:07.355650 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a2f23-db14-4aa2-add0-e1e09f27ee67 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:07.358086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-637a2f23-db14-4aa2-add0-e1e09f27ee67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:07.411289 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6592ab-bda3-4e8b-a6c8-06460eb5322b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:07.446370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a2f23-db14-4aa2-add0-e1e09f27ee67 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:07.452622 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51964d7-fb90-4b61-b89e-62c9560678c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:07.453264 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51964d7-fb90-4b61-b89e-62c9560678c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:07.453541 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51964d7-fb90-4b61-b89e-62c9560678c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:07.453810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51964d7-fb90-4b61-b89e-62c9560678c3 admin admin] PATH_INFO: `/v3/users/3e56a6388bbf4d10bac28c21c56273ea` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:07.455053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51964d7-fb90-4b61-b89e-62c9560678c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e56a6388bbf4d10bac28c21c56273ea)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:07.457039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51964d7-fb90-4b61-b89e-62c9560678c3 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:07.514274 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6592ab-bda3-4e8b-a6c8-06460eb5322b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:07.518301 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f51964d7-fb90-4b61-b89e-62c9560678c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e56a6388bbf4d10bac28c21c56273ea'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:07.520095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e7596-01be-49f6-8a48-5eda521f5955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:07.521077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e7596-01be-49f6-8a48-5eda521f5955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:07.521525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e7596-01be-49f6-8a48-5eda521f5955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:07.522063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e7596-01be-49f6-8a48-5eda521f5955 admin admin] PATH_INFO: `/v3/users/59cfcf9864444a76885768b3a4fd5842` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:07.523477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e7596-01be-49f6-8a48-5eda521f5955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59cfcf9864444a76885768b3a4fd5842)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:07.525556 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e7596-01be-49f6-8a48-5eda521f5955 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:07.529039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:07.540953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f51964d7-fb90-4b61-b89e-62c9560678c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e56a6388bbf4d10bac28c21c56273ea'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:07.551828 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f51964d7-fb90-4b61-b89e-62c9560678c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e56a6388bbf4d10bac28c21c56273ea'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:07.566573 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f51964d7-fb90-4b61-b89e-62c9560678c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e56a6388bbf4d10bac28c21c56273ea'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:07.567551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-f51964d7-fb90-4b61-b89e-62c9560678c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e56a6388bbf4d10bac28c21c56273ea'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:07.578668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1205/2442] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:07 2022] DELETE /identity/v3/users/3e56a6388bbf4d10bac28c21c56273ea => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:07.580257 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10def2e-a8c6-48f2-929d-136b1ddeef8e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:07.580618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10def2e-a8c6-48f2-929d-136b1ddeef8e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:07.580985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10def2e-a8c6-48f2-929d-136b1ddeef8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:07.606391 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0e6e7596-01be-49f6-8a48-5eda521f5955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59cfcf9864444a76885768b3a4fd5842'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:07.606658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-f10def2e-a8c6-48f2-929d-136b1ddeef8e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:07.608575 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f10def2e-a8c6-48f2-929d-136b1ddeef8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:07.621242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0e6e7596-01be-49f6-8a48-5eda521f5955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59cfcf9864444a76885768b3a4fd5842'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:07.622169 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0e6e7596-01be-49f6-8a48-5eda521f5955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59cfcf9864444a76885768b3a4fd5842'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:07.638121 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:07.646465 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0e6e7596-01be-49f6-8a48-5eda521f5955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59cfcf9864444a76885768b3a4fd5842'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:07.656884 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0e6e7596-01be-49f6-8a48-5eda521f5955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59cfcf9864444a76885768b3a4fd5842'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:07.664907 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1238/2443] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:07 2022] DELETE /identity/v3/users/59cfcf9864444a76885768b3a4fd5842 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:07.702276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1206/2444] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:07.713289 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e7596-01be-49f6-8a48-5eda521f5955 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:07.715668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0e6e7596-01be-49f6-8a48-5eda521f5955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:07.804348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e7596-01be-49f6-8a48-5eda521f5955 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:07.809145 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100ca2f-938b-4f95-aff4-a4784a60ce06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:07.809769 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100ca2f-938b-4f95-aff4-a4784a60ce06 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:07.810046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100ca2f-938b-4f95-aff4-a4784a60ce06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:07.810348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100ca2f-938b-4f95-aff4-a4784a60ce06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:07.848910 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10def2e-a8c6-48f2-929d-136b1ddeef8e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:07.898049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100ca2f-938b-4f95-aff4-a4784a60ce06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:07.900750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100ca2f-938b-4f95-aff4-a4784a60ce06 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:07.921813 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1239/2445] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:07.924468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100ca2f-938b-4f95-aff4-a4784a60ce06 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:07.927045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a100ca2f-938b-4f95-aff4-a4784a60ce06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:07.943316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10def2e-a8c6-48f2-929d-136b1ddeef8e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:07.949117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eedde1-bce9-4a92-80d0-a4e43d99b558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:07.949957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eedde1-bce9-4a92-80d0-a4e43d99b558 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:07.950399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eedde1-bce9-4a92-80d0-a4e43d99b558 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:07.950850 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eedde1-bce9-4a92-80d0-a4e43d99b558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:07.953140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-44eedde1-bce9-4a92-80d0-a4e43d99b558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:07.953349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100ca2f-938b-4f95-aff4-a4784a60ce06 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:07.964705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:07.966102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:07.966598 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:07.967060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] PATH_INFO: `/v3/projects/1555e6f3a9f04390b4d021c5d64c5cfa` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:07.978128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1555e6f3a9f04390b4d021c5d64c5cfa)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:07.982309 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:08.038007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:08.066684 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1555e6f3a9f04390b4d021c5d64c5cfa', 'request_id': 'req-62a07caa-a946-4133-a5ff-fcf5d34cb698'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.078550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1555e6f3a9f04390b4d021c5d64c5cfa', 'request_id': 'req-62a07caa-a946-4133-a5ff-fcf5d34cb698'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.081865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eedde1-bce9-4a92-80d0-a4e43d99b558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:08.083584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eedde1-bce9-4a92-80d0-a4e43d99b558 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:08.092539 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1555e6f3a9f04390b4d021c5d64c5cfa', 'request_id': 'req-62a07caa-a946-4133-a5ff-fcf5d34cb698'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.100202 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1207/2446] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:08.114835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] The token cache is being invalidate because project 1555e6f3a9f04390b4d021c5d64c5cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:08.115130 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.116325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.119718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1240/2447] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:07 2022] DELETE /identity/v3/projects/1555e6f3a9f04390b4d021c5d64c5cfa => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:08.125333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eedde1-bce9-4a92-80d0-a4e43d99b558 service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:08.140271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eedde1-bce9-4a92-80d0-a4e43d99b558 service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:08.219026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e3db0-9dec-4442-91e3-039345b19727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:08.220033 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e3db0-9dec-4442-91e3-039345b19727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:08.220481 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e3db0-9dec-4442-91e3-039345b19727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:08.220915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e3db0-9dec-4442-91e3-039345b19727 admin admin] PATH_INFO: `/v3/users/eca8248d524244e0b4ef0f4fa2ff3470` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:08.222847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e3db0-9dec-4442-91e3-039345b19727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca8248d524244e0b4ef0f4fa2ff3470)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:08.226712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e3db0-9dec-4442-91e3-039345b19727 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:08.258107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:08.297612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a12e3db0-9dec-4442-91e3-039345b19727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca8248d524244e0b4ef0f4fa2ff3470'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.316450 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a12e3db0-9dec-4442-91e3-039345b19727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca8248d524244e0b4ef0f4fa2ff3470'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.328251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a12e3db0-9dec-4442-91e3-039345b19727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca8248d524244e0b4ef0f4fa2ff3470'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.344921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a12e3db0-9dec-4442-91e3-039345b19727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca8248d524244e0b4ef0f4fa2ff3470'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.346545 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a12e3db0-9dec-4442-91e3-039345b19727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca8248d524244e0b4ef0f4fa2ff3470'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.359589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1208/2448] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:08 2022] DELETE /identity/v3/users/eca8248d524244e0b4ef0f4fa2ff3470 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:08.552294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:08.558741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:08.580341 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a07caa-a946-4133-a5ff-fcf5d34cb698 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:08.648594 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e3db0-9dec-4442-91e3-039345b19727 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:08.650389 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a12e3db0-9dec-4442-91e3-039345b19727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:08.692813 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:08.694372 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:08.694718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:08.695055 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] PATH_INFO: `/v3/projects/da1df3c6980a495fa5344417e8d8cdd9` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:08.697163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=da1df3c6980a495fa5344417e8d8cdd9)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:08.700211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:08.778369 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da1df3c6980a495fa5344417e8d8cdd9', 'request_id': 'req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.786991 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da1df3c6980a495fa5344417e8d8cdd9', 'request_id': 'req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.798350 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da1df3c6980a495fa5344417e8d8cdd9', 'request_id': 'req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.798619 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e3db0-9dec-4442-91e3-039345b19727 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:08.824885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] The token cache is being invalidate because project da1df3c6980a495fa5344417e8d8cdd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:08.825325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.826600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:08.833503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1241/2449] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:08 2022] DELETE /identity/v3/projects/da1df3c6980a495fa5344417e8d8cdd9 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:08.835381 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:08.838744 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:08.906602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092f5f8-0ec6-4953-8297-eefb688b2d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:08.907974 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092f5f8-0ec6-4953-8297-eefb688b2d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:08.908648 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092f5f8-0ec6-4953-8297-eefb688b2d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:08.909086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092f5f8-0ec6-4953-8297-eefb688b2d7c admin admin] PATH_INFO: `/v3/projects/c334c68a88084957a49432fd95d4e165` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:08.911000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092f5f8-0ec6-4953-8297-eefb688b2d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c334c68a88084957a49432fd95d4e165)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:08.914256 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092f5f8-0ec6-4953-8297-eefb688b2d7c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:08.921586 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:08.968317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565e974-b64c-46b3-bdc6-e347d8ff1a3d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:08.975339 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c392e1-dc07-4886-a3f1-c5432fb1a41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:08.976204 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c392e1-dc07-4886-a3f1-c5432fb1a41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:08.977063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c392e1-dc07-4886-a3f1-c5432fb1a41c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:08.977765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c392e1-dc07-4886-a3f1-c5432fb1a41c admin admin] PATH_INFO: `/v3/users/9bf112f199194baeb2840d2cec9c7229` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:08.979799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c392e1-dc07-4886-a3f1-c5432fb1a41c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf112f199194baeb2840d2cec9c7229)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:08.983031 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c392e1-dc07-4886-a3f1-c5432fb1a41c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:09.008074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:09.011408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d092f5f8-0ec6-4953-8297-eefb688b2d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c334c68a88084957a49432fd95d4e165', 'request_id': 'req-d092f5f8-0ec6-4953-8297-eefb688b2d7c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.033213 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d092f5f8-0ec6-4953-8297-eefb688b2d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c334c68a88084957a49432fd95d4e165', 'request_id': 'req-d092f5f8-0ec6-4953-8297-eefb688b2d7c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.046726 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d092f5f8-0ec6-4953-8297-eefb688b2d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c334c68a88084957a49432fd95d4e165', 'request_id': 'req-d092f5f8-0ec6-4953-8297-eefb688b2d7c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.064877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d092f5f8-0ec6-4953-8297-eefb688b2d7c admin admin] The token cache is being invalidate because project c334c68a88084957a49432fd95d4e165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:09.065235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d092f5f8-0ec6-4953-8297-eefb688b2d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.066222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d092f5f8-0ec6-4953-8297-eefb688b2d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.067793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1209/2450] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:08 2022] DELETE /identity/v3/projects/c334c68a88084957a49432fd95d4e165 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:09.074258 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d9b6e-6a7e-47d3-807d-e66b4c8f47d0 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:09.074906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d9b6e-6a7e-47d3-807d-e66b4c8f47d0 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:09.075466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d9b6e-6a7e-47d3-807d-e66b4c8f47d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:09.105275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5c392e1-dc07-4886-a3f1-c5432fb1a41c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf112f199194baeb2840d2cec9c7229'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.108905 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-0b8d9b6e-6a7e-47d3-807d-e66b4c8f47d0 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:09.110338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0b8d9b6e-6a7e-47d3-807d-e66b4c8f47d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:09.117711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5c392e1-dc07-4886-a3f1-c5432fb1a41c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf112f199194baeb2840d2cec9c7229'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.118676 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5c392e1-dc07-4886-a3f1-c5432fb1a41c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf112f199194baeb2840d2cec9c7229'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.134626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5c392e1-dc07-4886-a3f1-c5432fb1a41c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf112f199194baeb2840d2cec9c7229'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.142521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5c392e1-dc07-4886-a3f1-c5432fb1a41c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf112f199194baeb2840d2cec9c7229'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.150909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1242/2451] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:08 2022] DELETE /identity/v3/users/9bf112f199194baeb2840d2cec9c7229 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:09.256691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1210/2452] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:09.269386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693dc0e-b4ed-4fd6-8afb-775f6b359326 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:09.269770 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693dc0e-b4ed-4fd6-8afb-775f6b359326 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:09.270117 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693dc0e-b4ed-4fd6-8afb-775f6b359326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:09.298389 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c693dc0e-b4ed-4fd6-8afb-775f6b359326 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:09.299738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c693dc0e-b4ed-4fd6-8afb-775f6b359326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:09.391418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1243/2453] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:09.396741 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d9b6e-6a7e-47d3-807d-e66b4c8f47d0 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:09.418197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d9b6e-6a7e-47d3-807d-e66b4c8f47d0 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:09.426310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96e944-0355-4ff8-bfa6-a6e231c8cff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:09.427340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96e944-0355-4ff8-bfa6-a6e231c8cff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:09.427974 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96e944-0355-4ff8-bfa6-a6e231c8cff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:09.428440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96e944-0355-4ff8-bfa6-a6e231c8cff9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:09.430141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96e944-0355-4ff8-bfa6-a6e231c8cff9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:09.430733 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96e944-0355-4ff8-bfa6-a6e231c8cff9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:09.433957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96e944-0355-4ff8-bfa6-a6e231c8cff9 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:09.457522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1211/2454] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:09.467427 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693dc0e-b4ed-4fd6-8afb-775f6b359326 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:09.481290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693dc0e-b4ed-4fd6-8afb-775f6b359326 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:09.486707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a00b2a-30fa-47da-bf9a-721ee62c585e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:09.487527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a00b2a-30fa-47da-bf9a-721ee62c585e admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:09.487665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a00b2a-30fa-47da-bf9a-721ee62c585e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:09.487922 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a00b2a-30fa-47da-bf9a-721ee62c585e admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:09.488554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a00b2a-30fa-47da-bf9a-721ee62c585e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:09.490471 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a00b2a-30fa-47da-bf9a-721ee62c585e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:09.522908 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1244/2455] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:09 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:09.528589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96e944-0355-4ff8-bfa6-a6e231c8cff9 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:09.539794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96e944-0355-4ff8-bfa6-a6e231c8cff9 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:09.625808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f083e1-2b14-4472-bb4b-5399509505cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:09.626597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f083e1-2b14-4472-bb4b-5399509505cd admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:09.629033 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f083e1-2b14-4472-bb4b-5399509505cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:09.629033 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f083e1-2b14-4472-bb4b-5399509505cd admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:09.629033 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f083e1-2b14-4472-bb4b-5399509505cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:09.632391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f083e1-2b14-4472-bb4b-5399509505cd admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:09.660007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1212/2456] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:09 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:09.665332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a00b2a-30fa-47da-bf9a-721ee62c585e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:09.684419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f083e1-2b14-4472-bb4b-5399509505cd admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:09.685892 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a00b2a-30fa-47da-bf9a-721ee62c585e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:09.686355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-54f083e1-2b14-4472-bb4b-5399509505cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:09.697006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c5587-8d74-45bd-a106-d3155b0b6a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:09.698358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c5587-8d74-45bd-a106-d3155b0b6a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:09.699074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c5587-8d74-45bd-a106-d3155b0b6a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:09.699779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c5587-8d74-45bd-a106-d3155b0b6a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:09.701030 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c5587-8d74-45bd-a106-d3155b0b6a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:09.702946 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f083e1-2b14-4472-bb4b-5399509505cd admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:09.704332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c5587-8d74-45bd-a106-d3155b0b6a99 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:09.710128 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03392972-ecb5-49e9-ad38-abdbe2964bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:09.710877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03392972-ecb5-49e9-ad38-abdbe2964bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:09.712030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03392972-ecb5-49e9-ad38-abdbe2964bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:09.712030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03392972-ecb5-49e9-ad38-abdbe2964bb2 admin admin] PATH_INFO: `/v3/projects/f1b35790f9dd4e5d9dfb6301460bd172` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:09.713233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03392972-ecb5-49e9-ad38-abdbe2964bb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b35790f9dd4e5d9dfb6301460bd172)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:09.715853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03392972-ecb5-49e9-ad38-abdbe2964bb2 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:09.721177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1245/2457] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:09.726568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c5587-8d74-45bd-a106-d3155b0b6a99 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:09.747622 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c5587-8d74-45bd-a106-d3155b0b6a99 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:09.767415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36810f77-865b-425b-8cf6-fae03ede5cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:09.768399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36810f77-865b-425b-8cf6-fae03ede5cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:09.768779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36810f77-865b-425b-8cf6-fae03ede5cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:09.769139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36810f77-865b-425b-8cf6-fae03ede5cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:09.769879 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36810f77-865b-425b-8cf6-fae03ede5cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:09.771686 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36810f77-865b-425b-8cf6-fae03ede5cb2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:09.783535 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1246/2458] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:09.787523 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36810f77-865b-425b-8cf6-fae03ede5cb2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:09.801408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36810f77-865b-425b-8cf6-fae03ede5cb2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:09.802480 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:09.824913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-03392972-ecb5-49e9-ad38-abdbe2964bb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b35790f9dd4e5d9dfb6301460bd172', 'request_id': 'req-03392972-ecb5-49e9-ad38-abdbe2964bb2'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.838529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:09.842753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-03392972-ecb5-49e9-ad38-abdbe2964bb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b35790f9dd4e5d9dfb6301460bd172', 'request_id': 'req-03392972-ecb5-49e9-ad38-abdbe2964bb2'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.860971 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-03392972-ecb5-49e9-ad38-abdbe2964bb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b35790f9dd4e5d9dfb6301460bd172', 'request_id': 'req-03392972-ecb5-49e9-ad38-abdbe2964bb2'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.906060 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-03392972-ecb5-49e9-ad38-abdbe2964bb2 admin admin] The token cache is being invalidate because project f1b35790f9dd4e5d9dfb6301460bd172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:09.906551 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-03392972-ecb5-49e9-ad38-abdbe2964bb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.918102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-03392972-ecb5-49e9-ad38-abdbe2964bb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:09.920640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1213/2459] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:09 2022] DELETE /identity/v3/projects/f1b35790f9dd4e5d9dfb6301460bd172 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:09.926000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea4f31-495d-4dc2-a451-48e453e0792a None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:09.926302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea4f31-495d-4dc2-a451-48e453e0792a None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:09.926580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea4f31-495d-4dc2-a451-48e453e0792a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:09.954865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-74ea4f31-495d-4dc2-a451-48e453e0792a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:09.956249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-74ea4f31-495d-4dc2-a451-48e453e0792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:09.972146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11c989-aa69-4ee4-90f8-071567454139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:09.973011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11c989-aa69-4ee4-90f8-071567454139 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:09.973384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11c989-aa69-4ee4-90f8-071567454139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:09.973748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11c989-aa69-4ee4-90f8-071567454139 admin admin] PATH_INFO: `/v3/projects/addd0840454b4b65b339e6e4561ef17a/users/497dcb087f094c61a400c2faf896b674/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:10.011621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11c989-aa69-4ee4-90f8-071567454139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=addd0840454b4b65b339e6e4561ef17a, user_id=497dcb087f094c61a400c2faf896b674, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:10.015532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11c989-aa69-4ee4-90f8-071567454139 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:10.069249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1247/2460] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:09 2022] PUT /identity/v3/projects/addd0840454b4b65b339e6e4561ef17a/users/497dcb087f094c61a400c2faf896b674/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:10.077445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ef58d-40aa-4912-b4be-afdc4aee38a8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:10.077835 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ef58d-40aa-4912-b4be-afdc4aee38a8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:10.078407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ef58d-40aa-4912-b4be-afdc4aee38a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:10.140015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1214/2461] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:10.152779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e38811-f7fb-4c51-8bc9-10e69f441464 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:10.153276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e38811-f7fb-4c51-8bc9-10e69f441464 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:10.153755 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e38811-f7fb-4c51-8bc9-10e69f441464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:10.155697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-470ef58d-40aa-4912-b4be-afdc4aee38a8 None None] MFA Rules not processed for user `497dcb087f094c61a400c2faf896b674`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:10.157694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-470ef58d-40aa-4912-b4be-afdc4aee38a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:10.177380 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-34e38811-f7fb-4c51-8bc9-10e69f441464 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:10.178629 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-34e38811-f7fb-4c51-8bc9-10e69f441464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:10.299564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1248/2462] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:10.305763 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ef58d-40aa-4912-b4be-afdc4aee38a8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:10.308448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-470ef58d-40aa-4912-b4be-afdc4aee38a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:10.320000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1215/2463] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:10.325471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e38811-f7fb-4c51-8bc9-10e69f441464 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:10.343993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e38811-f7fb-4c51-8bc9-10e69f441464 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:10.366936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cd11a-c5ec-495f-8db3-d19ae92757cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:10.368212 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cd11a-c5ec-495f-8db3-d19ae92757cc admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:10.369431 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cd11a-c5ec-495f-8db3-d19ae92757cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:10.370464 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cd11a-c5ec-495f-8db3-d19ae92757cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:10.372316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cd11a-c5ec-495f-8db3-d19ae92757cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:10.373176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cd11a-c5ec-495f-8db3-d19ae92757cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:10.376362 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cd11a-c5ec-495f-8db3-d19ae92757cc admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:10.397621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1216/2464] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:43:10.398759 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cd11a-c5ec-495f-8db3-d19ae92757cc admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:10.412643 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cd11a-c5ec-495f-8db3-d19ae92757cc admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:10.417823 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ef58d-40aa-4912-b4be-afdc4aee38a8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:10.419071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f180f-a48c-4db2-9dd6-72b805880038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:10.420032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f180f-a48c-4db2-9dd6-72b805880038 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:10.420314 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f180f-a48c-4db2-9dd6-72b805880038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:10.420618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f180f-a48c-4db2-9dd6-72b805880038 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:10.421367 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f180f-a48c-4db2-9dd6-72b805880038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:10.424141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f180f-a48c-4db2-9dd6-72b805880038 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:10.425688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01118383-d714-4603-acd6-6ab143f67c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:10.425688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01118383-d714-4603-acd6-6ab143f67c57 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:10.426216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01118383-d714-4603-acd6-6ab143f67c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:10.426782 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01118383-d714-4603-acd6-6ab143f67c57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:10.428081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01118383-d714-4603-acd6-6ab143f67c57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:10.432203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01118383-d714-4603-acd6-6ab143f67c57 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:10.466840 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1217/2465] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:10 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:10.468447 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f180f-a48c-4db2-9dd6-72b805880038 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:10.485435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1249/2466] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:10 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:10.493134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01118383-d714-4603-acd6-6ab143f67c57 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:10.519466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:10.534690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:10.561116 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f180f-a48c-4db2-9dd6-72b805880038 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:10.585807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01118383-d714-4603-acd6-6ab143f67c57 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:10.593659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b86d0-ce71-46b4-9046-887ddb0e4c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:10.594210 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b86d0-ce71-46b4-9046-887ddb0e4c2d admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:10.594510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b86d0-ce71-46b4-9046-887ddb0e4c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:10.594830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b86d0-ce71-46b4-9046-887ddb0e4c2d admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:10.595408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b86d0-ce71-46b4-9046-887ddb0e4c2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:10.597325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b86d0-ce71-46b4-9046-887ddb0e4c2d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:10.625573 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1250/2467] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:10 2022] POST /identity/v3/users => generated 298 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:10.630652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b86d0-ce71-46b4-9046-887ddb0e4c2d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:10.643786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b86d0-ce71-46b4-9046-887ddb0e4c2d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:10.647033 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081e1ab-5e76-40f3-9d0f-98a50cd8b49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:10.648211 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081e1ab-5e76-40f3-9d0f-98a50cd8b49e admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:10.648785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081e1ab-5e76-40f3-9d0f-98a50cd8b49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:10.649362 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081e1ab-5e76-40f3-9d0f-98a50cd8b49e admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:10.649800 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e1182-467f-4b77-8086-f0f9856ebef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:10.650466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e1182-467f-4b77-8086-f0f9856ebef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:10.650715 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081e1ab-5e76-40f3-9d0f-98a50cd8b49e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:10.650851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e1182-467f-4b77-8086-f0f9856ebef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:10.651042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e1182-467f-4b77-8086-f0f9856ebef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:10.651810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e1182-467f-4b77-8086-f0f9856ebef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:10.655881 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e1182-467f-4b77-8086-f0f9856ebef7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:10.656072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081e1ab-5e76-40f3-9d0f-98a50cd8b49e admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:10.668005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1251/2468] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:10.673008 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e1182-467f-4b77-8086-f0f9856ebef7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:10.686984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1218/2469] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:10 2022] POST /identity/v3/users => generated 292 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:10.690165 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e1182-467f-4b77-8086-f0f9856ebef7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:10.692942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081e1ab-5e76-40f3-9d0f-98a50cd8b49e admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:10.698644 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a2aad-87df-4750-b335-cc706b2f01f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:10.699751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a2aad-87df-4750-b335-cc706b2f01f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:10.700261 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a2aad-87df-4750-b335-cc706b2f01f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:10.701249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a2aad-87df-4750-b335-cc706b2f01f1 admin admin] PATH_INFO: `/v3/projects/fe9f14c0063e42c8bff05f353d3d331b/users/216b734268434b3db0c386be140416a0/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:10.705481 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081e1ab-5e76-40f3-9d0f-98a50cd8b49e admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:10.717807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44255d-f3cc-4128-b833-faff26b9fe16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:10.718596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44255d-f3cc-4128-b833-faff26b9fe16 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:10.718969 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44255d-f3cc-4128-b833-faff26b9fe16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:10.721702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44255d-f3cc-4128-b833-faff26b9fe16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:10.721702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44255d-f3cc-4128-b833-faff26b9fe16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:10.722596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44255d-f3cc-4128-b833-faff26b9fe16 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:10.741836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1219/2470] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:10.744256 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44255d-f3cc-4128-b833-faff26b9fe16 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:10.769789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44255d-f3cc-4128-b833-faff26b9fe16 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:10.780164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b7db0-7d2c-4b31-930f-13966cbf6c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:10.781197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b7db0-7d2c-4b31-930f-13966cbf6c28 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:10.781690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b7db0-7d2c-4b31-930f-13966cbf6c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:10.782184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b7db0-7d2c-4b31-930f-13966cbf6c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:10.782947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b7db0-7d2c-4b31-930f-13966cbf6c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:10.785471 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b7db0-7d2c-4b31-930f-13966cbf6c28 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:10.803931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1220/2471] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:10.809892 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b7db0-7d2c-4b31-930f-13966cbf6c28 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:10.825154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b7db0-7d2c-4b31-930f-13966cbf6c28 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:10.833499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35fed7-dc9b-46de-81f5-18dc61963e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:10.834502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35fed7-dc9b-46de-81f5-18dc61963e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:10.835019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35fed7-dc9b-46de-81f5-18dc61963e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:10.835529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35fed7-dc9b-46de-81f5-18dc61963e68 admin admin] PATH_INFO: `/v3/projects/8ef2577c3b084c31940158a84af6f3f2/users/25195d1955c04da1861bb519f405b031/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:10.858487 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a2aad-87df-4750-b335-cc706b2f01f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9f14c0063e42c8bff05f353d3d331b, user_id=216b734268434b3db0c386be140416a0, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:10.861942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a2aad-87df-4750-b335-cc706b2f01f1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:10.863097 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d35fed7-dc9b-46de-81f5-18dc61963e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef2577c3b084c31940158a84af6f3f2, user_id=25195d1955c04da1861bb519f405b031, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:10.869000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d35fed7-dc9b-46de-81f5-18dc61963e68 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:10.886948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1252/2472] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:10 2022] PUT /identity/v3/projects/fe9f14c0063e42c8bff05f353d3d331b/users/216b734268434b3db0c386be140416a0/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:10.895854 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1221/2473] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:10 2022] PUT /identity/v3/projects/8ef2577c3b084c31940158a84af6f3f2/users/25195d1955c04da1861bb519f405b031/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:10.896087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a2aad-87df-4750-b335-cc706b2f01f1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:10.902834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac21339-2e5b-4226-9732-0c5bf2f625c2 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:10.903456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac21339-2e5b-4226-9732-0c5bf2f625c2 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:10.903800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac21339-2e5b-4226-9732-0c5bf2f625c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:10.941269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-bac21339-2e5b-4226-9732-0c5bf2f625c2 None None] MFA Rules not processed for user `25195d1955c04da1861bb519f405b031`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:10.944327 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-bac21339-2e5b-4226-9732-0c5bf2f625c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:10.985948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a2aad-87df-4750-b335-cc706b2f01f1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:11.054932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1222/2474] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:11.062188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac21339-2e5b-4226-9732-0c5bf2f625c2 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:11.094522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99656c1-b28e-4aaa-8ef3-304e202d770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:11.095243 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99656c1-b28e-4aaa-8ef3-304e202d770b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:11.095579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99656c1-b28e-4aaa-8ef3-304e202d770b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:11.095865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99656c1-b28e-4aaa-8ef3-304e202d770b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:11.096574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99656c1-b28e-4aaa-8ef3-304e202d770b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:11.098290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99656c1-b28e-4aaa-8ef3-304e202d770b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:11.107765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:11.110332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1253/2475] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:11.115932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99656c1-b28e-4aaa-8ef3-304e202d770b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:11.132669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99656c1-b28e-4aaa-8ef3-304e202d770b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:11.140044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d80a1-940b-46d5-9896-1a1d06cdc75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:11.141127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d80a1-940b-46d5-9896-1a1d06cdc75e admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:11.141575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d80a1-940b-46d5-9896-1a1d06cdc75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:11.142260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d80a1-940b-46d5-9896-1a1d06cdc75e admin admin] PATH_INFO: `/v3/projects/fe9f14c0063e42c8bff05f353d3d331b/users/216b734268434b3db0c386be140416a0/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:11.146352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d80a1-940b-46d5-9896-1a1d06cdc75e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9f14c0063e42c8bff05f353d3d331b, user_id=216b734268434b3db0c386be140416a0, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:11.150673 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d80a1-940b-46d5-9896-1a1d06cdc75e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:11.168153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1254/2476] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:11 2022] PUT /identity/v3/projects/fe9f14c0063e42c8bff05f353d3d331b/users/216b734268434b3db0c386be140416a0/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:11.173591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac21339-2e5b-4226-9732-0c5bf2f625c2 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:11.180113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2968-09d3-49c0-ae3c-ae7df2cc8adf None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:11.180395 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2968-09d3-49c0-ae3c-ae7df2cc8adf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:11.180765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2968-09d3-49c0-ae3c-ae7df2cc8adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:11.188449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:11.216174 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f7dc2968-09d3-49c0-ae3c-ae7df2cc8adf None None] MFA Rules not processed for user `216b734268434b3db0c386be140416a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:11.217407 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f7dc2968-09d3-49c0-ae3c-ae7df2cc8adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:11.313728 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18449b-23be-499e-8ff6-a218614983b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:11.315094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18449b-23be-499e-8ff6-a218614983b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:11.315462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18449b-23be-499e-8ff6-a218614983b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:11.315794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18449b-23be-499e-8ff6-a218614983b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:11.316501 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18449b-23be-499e-8ff6-a218614983b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:11.318462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18449b-23be-499e-8ff6-a218614983b0 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:11.346574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1255/2477] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:11.355743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc2968-09d3-49c0-ae3c-ae7df2cc8adf None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:11.360697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f7dc2968-09d3-49c0-ae3c-ae7df2cc8adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:11.420508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1223/2478] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:11 2022] POST /identity/v3/projects => generated 381 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:11.424739 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18449b-23be-499e-8ff6-a218614983b0 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:11.463192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc2968-09d3-49c0-ae3c-ae7df2cc8adf None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:11.545482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7b5cd-dac0-486a-b5e5-01e8737de965 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:11.546359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7b5cd-dac0-486a-b5e5-01e8737de965 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:11.546831 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7b5cd-dac0-486a-b5e5-01e8737de965 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:11.547522 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7b5cd-dac0-486a-b5e5-01e8737de965 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:11.569235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18449b-23be-499e-8ff6-a218614983b0 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:11.578039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b3672-7a88-4357-9905-c3df25b019bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:11.578784 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b3672-7a88-4357-9905-c3df25b019bd admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:11.579282 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b3672-7a88-4357-9905-c3df25b019bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:11.579810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b3672-7a88-4357-9905-c3df25b019bd admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:11.581398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b3672-7a88-4357-9905-c3df25b019bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:11.584906 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b3672-7a88-4357-9905-c3df25b019bd admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:11.616699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1224/2479] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:11 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:11.622017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b3672-7a88-4357-9905-c3df25b019bd admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:11.633330 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b3672-7a88-4357-9905-c3df25b019bd admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:11.639292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309a04c-fda9-40b0-bcc1-77bc9527d95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:11.639882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309a04c-fda9-40b0-bcc1-77bc9527d95c admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:11.640215 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309a04c-fda9-40b0-bcc1-77bc9527d95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:11.640488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309a04c-fda9-40b0-bcc1-77bc9527d95c admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:11.641151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309a04c-fda9-40b0-bcc1-77bc9527d95c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:11.642766 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309a04c-fda9-40b0-bcc1-77bc9527d95c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:11.657394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1225/2480] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:11.666254 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309a04c-fda9-40b0-bcc1-77bc9527d95c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:11.668428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:11.676027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d7b5cd-dac0-486a-b5e5-01e8737de965 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:11.678358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d7b5cd-dac0-486a-b5e5-01e8737de965 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:11.683836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309a04c-fda9-40b0-bcc1-77bc9527d95c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:11.691299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43910b7c-8545-4818-b29e-b8d15b876ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:11.692380 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43910b7c-8545-4818-b29e-b8d15b876ffa admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:11.692834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43910b7c-8545-4818-b29e-b8d15b876ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:11.693356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43910b7c-8545-4818-b29e-b8d15b876ffa admin admin] PATH_INFO: `/v3/projects/db9a3a8e3237470d8a5d0f49d71a00b9/users/ff131d1a08734d599e4c41ed0cae06f2/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:11.696425 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1256/2481] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:11.731625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43910b7c-8545-4818-b29e-b8d15b876ffa admin admin] RBAC: Authorizing `identity:create_grant(project_id=db9a3a8e3237470d8a5d0f49d71a00b9, user_id=ff131d1a08734d599e4c41ed0cae06f2, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:11.750369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43910b7c-8545-4818-b29e-b8d15b876ffa admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:11.788081 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1226/2482] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:11 2022] PUT /identity/v3/projects/db9a3a8e3237470d8a5d0f49d71a00b9/users/ff131d1a08734d599e4c41ed0cae06f2/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:11.792262 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7b5cd-dac0-486a-b5e5-01e8737de965 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:11.813195 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7b5cd-dac0-486a-b5e5-01e8737de965 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:11.813780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43910b7c-8545-4818-b29e-b8d15b876ffa admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:11.828663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:11.920742 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43910b7c-8545-4818-b29e-b8d15b876ffa admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:11.928552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2f7ea-f7b8-4515-a90b-7ba9ed4aa112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:11.929723 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93d64c-013a-407c-be29-2f12a336f2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:11.929916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f7ea-f7b8-4515-a90b-7ba9ed4aa112 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:11.930068 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f7ea-f7b8-4515-a90b-7ba9ed4aa112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:11.931386 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93d64c-013a-407c-be29-2f12a336f2c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:11.931596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f7ea-f7b8-4515-a90b-7ba9ed4aa112 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:11.931758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93d64c-013a-407c-be29-2f12a336f2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:11.931758 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93d64c-013a-407c-be29-2f12a336f2c6 admin admin] PATH_INFO: `/v3/users/497dcb087f094c61a400c2faf896b674` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:11.932170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2f7ea-f7b8-4515-a90b-7ba9ed4aa112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:11.933539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93d64c-013a-407c-be29-2f12a336f2c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=497dcb087f094c61a400c2faf896b674)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:11.935399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2f7ea-f7b8-4515-a90b-7ba9ed4aa112 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:11.935930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93d64c-013a-407c-be29-2f12a336f2c6 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:11.951304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1257/2483] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:11.955935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2f7ea-f7b8-4515-a90b-7ba9ed4aa112 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:11.978586 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2f7ea-f7b8-4515-a90b-7ba9ed4aa112 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:11.986888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db086c-8fa4-407f-9819-5a8520b6f1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:11.988190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db086c-8fa4-407f-9819-5a8520b6f1f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:11.988760 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db086c-8fa4-407f-9819-5a8520b6f1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:11.989516 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db086c-8fa4-407f-9819-5a8520b6f1f8 admin admin] PATH_INFO: `/v3/projects/db9a3a8e3237470d8a5d0f49d71a00b9/users/ff131d1a08734d599e4c41ed0cae06f2/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:11.993480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db086c-8fa4-407f-9819-5a8520b6f1f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db9a3a8e3237470d8a5d0f49d71a00b9, user_id=ff131d1a08734d599e4c41ed0cae06f2, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:11.998361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db086c-8fa4-407f-9819-5a8520b6f1f8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:12.016810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9f93d64c-013a-407c-be29-2f12a336f2c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '497dcb087f094c61a400c2faf896b674'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.018417 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1258/2484] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:11 2022] PUT /identity/v3/projects/db9a3a8e3237470d8a5d0f49d71a00b9/users/ff131d1a08734d599e4c41ed0cae06f2/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:12.030616 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a61c56-9bc7-443f-8607-4bda6e071616 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:12.030898 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a61c56-9bc7-443f-8607-4bda6e071616 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:12.031197 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a61c56-9bc7-443f-8607-4bda6e071616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:12.040969 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9f93d64c-013a-407c-be29-2f12a336f2c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '497dcb087f094c61a400c2faf896b674'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.072092 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9f93d64c-013a-407c-be29-2f12a336f2c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '497dcb087f094c61a400c2faf896b674'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.077073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b3a61c56-9bc7-443f-8607-4bda6e071616 None None] MFA Rules not processed for user `ff131d1a08734d599e4c41ed0cae06f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:12.079757 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b3a61c56-9bc7-443f-8607-4bda6e071616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:12.097785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9f93d64c-013a-407c-be29-2f12a336f2c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '497dcb087f094c61a400c2faf896b674'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.098593 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9f93d64c-013a-407c-be29-2f12a336f2c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '497dcb087f094c61a400c2faf896b674'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:12.106537 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1227/2485] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:11 2022] DELETE /identity/v3/users/497dcb087f094c61a400c2faf896b674 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:12.111387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc4cbb-e613-41ae-bacf-ed979ddc1d8e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:12.111816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc4cbb-e613-41ae-bacf-ed979ddc1d8e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:12.112214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc4cbb-e613-41ae-bacf-ed979ddc1d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:12.134235 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-96cc4cbb-e613-41ae-bacf-ed979ddc1d8e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:12.135822 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-96cc4cbb-e613-41ae-bacf-ed979ddc1d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:12.192684 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1259/2486] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:12.204338 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a61c56-9bc7-443f-8607-4bda6e071616 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:12.205813 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b3a61c56-9bc7-443f-8607-4bda6e071616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:12.225423 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1228/2487] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:12.237112 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc4cbb-e613-41ae-bacf-ed979ddc1d8e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:12.240626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-96cc4cbb-e613-41ae-bacf-ed979ddc1d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:12.293991 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:12.295784 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a61c56-9bc7-443f-8607-4bda6e071616 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:12.315394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:12.361084 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc4cbb-e613-41ae-bacf-ed979ddc1d8e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:12.365940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238d50a-8839-495e-865f-252dcbb2fda0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:12.366745 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238d50a-8839-495e-865f-252dcbb2fda0 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:12.367174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238d50a-8839-495e-865f-252dcbb2fda0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:12.367611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238d50a-8839-495e-865f-252dcbb2fda0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:12.370793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4a23a-54b5-462b-98cf-0d9fb978da9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:12.371460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4a23a-54b5-462b-98cf-0d9fb978da9d service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:12.371747 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4a23a-54b5-462b-98cf-0d9fb978da9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:12.372072 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4a23a-54b5-462b-98cf-0d9fb978da9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:12.380874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238d50a-8839-495e-865f-252dcbb2fda0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:12.383097 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238d50a-8839-495e-865f-252dcbb2fda0 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:12.402808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1229/2488] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:12.515168 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4a23a-54b5-462b-98cf-0d9fb978da9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:12.519344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4a23a-54b5-462b-98cf-0d9fb978da9d service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:12.543334 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1260/2489] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:12.647414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238d50a-8839-495e-865f-252dcbb2fda0 service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:12.665077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238d50a-8839-495e-865f-252dcbb2fda0 service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:12.671722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024bc86-2db1-428a-b796-61a3635ec1c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:12.672828 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024bc86-2db1-428a-b796-61a3635ec1c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:12.676035 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024bc86-2db1-428a-b796-61a3635ec1c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:12.676673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024bc86-2db1-428a-b796-61a3635ec1c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:12.680827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9024bc86-2db1-428a-b796-61a3635ec1c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:12.781625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024bc86-2db1-428a-b796-61a3635ec1c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:12.783101 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024bc86-2db1-428a-b796-61a3635ec1c3 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:12.795454 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1230/2490] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:12.884187 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4a23a-54b5-462b-98cf-0d9fb978da9d service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:12.887100 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8cf4a23a-54b5-462b-98cf-0d9fb978da9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:12.930385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4a23a-54b5-462b-98cf-0d9fb978da9d service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:12.941564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:12.942877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:12.942877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:12.943666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] PATH_INFO: `/v3/projects/addd0840454b4b65b339e6e4561ef17a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:12.948011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] RBAC: Authorizing `identity:delete_project(project_id=addd0840454b4b65b339e6e4561ef17a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:12.950925 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:13.106313 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'addd0840454b4b65b339e6e4561ef17a', 'request_id': 'req-63d05a01-9899-4be3-a08c-2e81380bd35f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.128149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'addd0840454b4b65b339e6e4561ef17a', 'request_id': 'req-63d05a01-9899-4be3-a08c-2e81380bd35f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.137134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'addd0840454b4b65b339e6e4561ef17a', 'request_id': 'req-63d05a01-9899-4be3-a08c-2e81380bd35f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.156566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] The token cache is being invalidate because project addd0840454b4b65b339e6e4561ef17a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:13.157001 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.157853 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.163630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1261/2491] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:12 2022] DELETE /identity/v3/projects/addd0840454b4b65b339e6e4561ef17a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:13.171538 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024bc86-2db1-428a-b796-61a3635ec1c3 service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:13.183775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024bc86-2db1-428a-b796-61a3635ec1c3 service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:13.275025 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2cac-1900-4a01-b6e5-c21005e3fa06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:13.275995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2cac-1900-4a01-b6e5-c21005e3fa06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.276601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2cac-1900-4a01-b6e5-c21005e3fa06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.276877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2cac-1900-4a01-b6e5-c21005e3fa06 admin admin] PATH_INFO: `/v3/users/216b734268434b3db0c386be140416a0` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.278932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d2cac-1900-4a01-b6e5-c21005e3fa06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=216b734268434b3db0c386be140416a0)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:13.281762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d2cac-1900-4a01-b6e5-c21005e3fa06 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:13.290093 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:13.292152 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:13.308987 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d05a01-9899-4be3-a08c-2e81380bd35f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:13.314481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002bf3f-5344-46f0-94c9-6149f458e89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:13.315437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002bf3f-5344-46f0-94c9-6149f458e89e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.315853 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002bf3f-5344-46f0-94c9-6149f458e89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.316393 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002bf3f-5344-46f0-94c9-6149f458e89e admin admin] PATH_INFO: `/v3/users/25195d1955c04da1861bb519f405b031` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.318583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002bf3f-5344-46f0-94c9-6149f458e89e admin admin] RBAC: Authorizing `identity:delete_user(user_id=25195d1955c04da1861bb519f405b031)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:13.323493 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002bf3f-5344-46f0-94c9-6149f458e89e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:13.349695 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-145d2cac-1900-4a01-b6e5-c21005e3fa06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '216b734268434b3db0c386be140416a0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.367692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-145d2cac-1900-4a01-b6e5-c21005e3fa06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '216b734268434b3db0c386be140416a0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.375233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:13.378309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-145d2cac-1900-4a01-b6e5-c21005e3fa06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '216b734268434b3db0c386be140416a0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.383399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:13.387945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c002bf3f-5344-46f0-94c9-6149f458e89e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25195d1955c04da1861bb519f405b031'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.396544 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-145d2cac-1900-4a01-b6e5-c21005e3fa06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '216b734268434b3db0c386be140416a0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.398304 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-145d2cac-1900-4a01-b6e5-c21005e3fa06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '216b734268434b3db0c386be140416a0'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.405806 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c002bf3f-5344-46f0-94c9-6149f458e89e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25195d1955c04da1861bb519f405b031'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.408025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c002bf3f-5344-46f0-94c9-6149f458e89e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25195d1955c04da1861bb519f405b031'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.414764 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1231/2492] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:13 2022] DELETE /identity/v3/users/216b734268434b3db0c386be140416a0 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:13.438316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c002bf3f-5344-46f0-94c9-6149f458e89e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25195d1955c04da1861bb519f405b031'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.452706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c002bf3f-5344-46f0-94c9-6149f458e89e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25195d1955c04da1861bb519f405b031'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.470623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1262/2493] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:13 2022] DELETE /identity/v3/users/25195d1955c04da1861bb519f405b031 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:13.481053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e6b39-6e5e-48f1-8399-07381a078b89 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.482241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e6b39-6e5e-48f1-8399-07381a078b89 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.482634 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e6b39-6e5e-48f1-8399-07381a078b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.530477 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6e7e6b39-6e5e-48f1-8399-07381a078b89 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:13.531614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6e7e6b39-6e5e-48f1-8399-07381a078b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:13.634016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1232/2494] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:13.658569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002bf3f-5344-46f0-94c9-6149f458e89e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:13.661054 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c002bf3f-5344-46f0-94c9-6149f458e89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:13.745054 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002bf3f-5344-46f0-94c9-6149f458e89e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:13.750195 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99471862-2210-43c7-b854-7da4dd075c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:13.751394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99471862-2210-43c7-b854-7da4dd075c73 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.751618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99471862-2210-43c7-b854-7da4dd075c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.752132 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99471862-2210-43c7-b854-7da4dd075c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.770879 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99471862-2210-43c7-b854-7da4dd075c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:13.773691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99471862-2210-43c7-b854-7da4dd075c73 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:13.792422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1263/2495] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:13.843399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e6b39-6e5e-48f1-8399-07381a078b89 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:13.845193 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6e7e6b39-6e5e-48f1-8399-07381a078b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:13.863505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e6b39-6e5e-48f1-8399-07381a078b89 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:13.876284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30208b2d-5976-461e-853b-ea712a4b2187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:13.877644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30208b2d-5976-461e-853b-ea712a4b2187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:13.878468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30208b2d-5976-461e-853b-ea712a4b2187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:13.878617 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30208b2d-5976-461e-853b-ea712a4b2187 admin admin] PATH_INFO: `/v3/projects/fe9f14c0063e42c8bff05f353d3d331b` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:13.880737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30208b2d-5976-461e-853b-ea712a4b2187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe9f14c0063e42c8bff05f353d3d331b)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:13.883957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30208b2d-5976-461e-853b-ea712a4b2187 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:13.958239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-30208b2d-5976-461e-853b-ea712a4b2187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe9f14c0063e42c8bff05f353d3d331b', 'request_id': 'req-30208b2d-5976-461e-853b-ea712a4b2187'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.967147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-30208b2d-5976-461e-853b-ea712a4b2187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe9f14c0063e42c8bff05f353d3d331b', 'request_id': 'req-30208b2d-5976-461e-853b-ea712a4b2187'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.978269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-30208b2d-5976-461e-853b-ea712a4b2187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe9f14c0063e42c8bff05f353d3d331b', 'request_id': 'req-30208b2d-5976-461e-853b-ea712a4b2187'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.996406 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-30208b2d-5976-461e-853b-ea712a4b2187 admin admin] The token cache is being invalidate because project fe9f14c0063e42c8bff05f353d3d331b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:13.996766 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-30208b2d-5976-461e-853b-ea712a4b2187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:13.997843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-30208b2d-5976-461e-853b-ea712a4b2187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:14.000138 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1233/2496] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:13 2022] DELETE /identity/v3/projects/fe9f14c0063e42c8bff05f353d3d331b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:14.006968 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4779e94-a88d-47de-bb1f-c027002cc87e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.007771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4779e94-a88d-47de-bb1f-c027002cc87e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.007771 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4779e94-a88d-47de-bb1f-c027002cc87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.037787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b4779e94-a88d-47de-bb1f-c027002cc87e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:14.039736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b4779e94-a88d-47de-bb1f-c027002cc87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:14.137997 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1264/2497] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:14.147064 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699382d-ebed-4811-aa14-ecd08f230b54 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.147520 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699382d-ebed-4811-aa14-ecd08f230b54 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.147981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699382d-ebed-4811-aa14-ecd08f230b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.191656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-f699382d-ebed-4811-aa14-ecd08f230b54 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:14.192206 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4779e94-a88d-47de-bb1f-c027002cc87e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.193957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f699382d-ebed-4811-aa14-ecd08f230b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:14.194851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b4779e94-a88d-47de-bb1f-c027002cc87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:14.211153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4779e94-a88d-47de-bb1f-c027002cc87e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.221502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1234/2498] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:14.223524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.224192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.224468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.224767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] PATH_INFO: `/v3/projects/8ef2577c3b084c31940158a84af6f3f2` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.226518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef2577c3b084c31940158a84af6f3f2)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.227090 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699382d-ebed-4811-aa14-ecd08f230b54 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.229493 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.238175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699382d-ebed-4811-aa14-ecd08f230b54 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.244989 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51d8de-81af-470c-9573-4d62dfebe67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.244989 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51d8de-81af-470c-9573-4d62dfebe67a admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.244989 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51d8de-81af-470c-9573-4d62dfebe67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.244989 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51d8de-81af-470c-9573-4d62dfebe67a admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.246724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51d8de-81af-470c-9573-4d62dfebe67a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:14.246724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51d8de-81af-470c-9573-4d62dfebe67a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.248877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51d8de-81af-470c-9573-4d62dfebe67a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.272040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1235/2499] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:14.277760 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51d8de-81af-470c-9573-4d62dfebe67a admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.293524 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51d8de-81af-470c-9573-4d62dfebe67a admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.317193 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef2577c3b084c31940158a84af6f3f2', 'request_id': 'req-677da778-346f-4440-9ff8-4e74c872bbb4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:14.332238 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef2577c3b084c31940158a84af6f3f2', 'request_id': 'req-677da778-346f-4440-9ff8-4e74c872bbb4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:14.336240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:14.345110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef2577c3b084c31940158a84af6f3f2', 'request_id': 'req-677da778-346f-4440-9ff8-4e74c872bbb4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:14.368345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:14.369368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] The token cache is being invalidate because project 8ef2577c3b084c31940158a84af6f3f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:14.369786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:14.371402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:14.374527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1265/2500] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:14 2022] DELETE /identity/v3/projects/8ef2577c3b084c31940158a84af6f3f2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:14.380027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.382451 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:14.386579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf65297-09fb-4a3f-807f-1a603d225850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.387351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf65297-09fb-4a3f-807f-1a603d225850 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.387740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf65297-09fb-4a3f-807f-1a603d225850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.388522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf65297-09fb-4a3f-807f-1a603d225850 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.390172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf65297-09fb-4a3f-807f-1a603d225850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.392376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf65297-09fb-4a3f-807f-1a603d225850 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.409653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677da778-346f-4440-9ff8-4e74c872bbb4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.418979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8b218-5fca-4ee6-bba4-c334bb4905ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.420282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8b218-5fca-4ee6-bba4-c334bb4905ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.420756 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8b218-5fca-4ee6-bba4-c334bb4905ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.421490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8b218-5fca-4ee6-bba4-c334bb4905ab admin admin] PATH_INFO: `/v3/users/ff131d1a08734d599e4c41ed0cae06f2` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.423775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8b218-5fca-4ee6-bba4-c334bb4905ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff131d1a08734d599e4c41ed0cae06f2)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.427747 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8b218-5fca-4ee6-bba4-c334bb4905ab admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.447999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1236/2501] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:14 2022] POST /identity/v3/projects => generated 381 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:14.461332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf65297-09fb-4a3f-807f-1a603d225850 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.480584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-dcf65297-09fb-4a3f-807f-1a603d225850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:14.543614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-83d8b218-5fca-4ee6-bba4-c334bb4905ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff131d1a08734d599e4c41ed0cae06f2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:14.553815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-83d8b218-5fca-4ee6-bba4-c334bb4905ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff131d1a08734d599e4c41ed0cae06f2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:14.554653 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-83d8b218-5fca-4ee6-bba4-c334bb4905ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff131d1a08734d599e4c41ed0cae06f2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:14.581817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-83d8b218-5fca-4ee6-bba4-c334bb4905ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff131d1a08734d599e4c41ed0cae06f2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:14.582645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf65297-09fb-4a3f-807f-1a603d225850 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.590809 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-83d8b218-5fca-4ee6-bba4-c334bb4905ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff131d1a08734d599e4c41ed0cae06f2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:14.592157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f2835-3712-4a39-b868-667271f21435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.592931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f2835-3712-4a39-b868-667271f21435 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.593237 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f2835-3712-4a39-b868-667271f21435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.593617 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f2835-3712-4a39-b868-667271f21435 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.594729 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f2835-3712-4a39-b868-667271f21435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.597524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1266/2502] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:14 2022] DELETE /identity/v3/users/ff131d1a08734d599e4c41ed0cae06f2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:14.598096 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f2835-3712-4a39-b868-667271f21435 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.600795 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8b218-5fca-4ee6-bba4-c334bb4905ab admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.602279 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-83d8b218-5fca-4ee6-bba4-c334bb4905ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:14.632117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1237/2503] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:14 2022] POST /identity/v3/users => generated 291 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:14.635993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f2835-3712-4a39-b868-667271f21435 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.638526 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8a3f2835-3712-4a39-b868-667271f21435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:14.657599 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f2835-3712-4a39-b868-667271f21435 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.704584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8b218-5fca-4ee6-bba4-c334bb4905ab admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.709267 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c9352-f849-4744-96f9-842053b5674e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.710072 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c9352-f849-4744-96f9-842053b5674e service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.710357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c9352-f849-4744-96f9-842053b5674e service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.710633 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c9352-f849-4744-96f9-842053b5674e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.712714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fa3c9352-f849-4744-96f9-842053b5674e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:14.742663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d308a97-9ef0-4170-a044-5611c8499683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.743859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d308a97-9ef0-4170-a044-5611c8499683 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.744369 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d308a97-9ef0-4170-a044-5611c8499683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.744858 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d308a97-9ef0-4170-a044-5611c8499683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.745561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d308a97-9ef0-4170-a044-5611c8499683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.747585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d308a97-9ef0-4170-a044-5611c8499683 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.772609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1238/2504] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:14.777625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d308a97-9ef0-4170-a044-5611c8499683 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.798364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d308a97-9ef0-4170-a044-5611c8499683 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.816049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740fda7-c236-4567-b379-b3d6084fcca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.817450 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740fda7-c236-4567-b379-b3d6084fcca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.818015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740fda7-c236-4567-b379-b3d6084fcca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.818433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740fda7-c236-4567-b379-b3d6084fcca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.819485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740fda7-c236-4567-b379-b3d6084fcca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.822329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c9352-f849-4744-96f9-842053b5674e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.822649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740fda7-c236-4567-b379-b3d6084fcca0 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.823063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c9352-f849-4744-96f9-842053b5674e service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.835799 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1239/2505] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:14.836582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1267/2506] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:14.840632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740fda7-c236-4567-b379-b3d6084fcca0 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:14.858705 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740fda7-c236-4567-b379-b3d6084fcca0 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:14.865178 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac641b4-4a02-4a61-a70b-7990646d69a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:14.866064 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac641b4-4a02-4a61-a70b-7990646d69a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.867827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac641b4-4a02-4a61-a70b-7990646d69a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.868676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac641b4-4a02-4a61-a70b-7990646d69a4 admin admin] PATH_INFO: `/v3/projects/665b6efdb4794e5abf46ef796bbbc169/users/590c66767782487682b601a7f07c8b65/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.902191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac641b4-4a02-4a61-a70b-7990646d69a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=665b6efdb4794e5abf46ef796bbbc169, user_id=590c66767782487682b601a7f07c8b65, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:14.906458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac641b4-4a02-4a61-a70b-7990646d69a4 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:14.933350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1240/2507] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:14 2022] PUT /identity/v3/projects/665b6efdb4794e5abf46ef796bbbc169/users/590c66767782487682b601a7f07c8b65/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:14.946443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcc678-3d87-4d2a-9c52-add813ca8acb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:14.947472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcc678-3d87-4d2a-9c52-add813ca8acb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:14.948131 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcc678-3d87-4d2a-9c52-add813ca8acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:14.987634 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:15.003945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-84dcc678-3d87-4d2a-9c52-add813ca8acb None None] MFA Rules not processed for user `590c66767782487682b601a7f07c8b65`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:15.006815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-84dcc678-3d87-4d2a-9c52-add813ca8acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:15.064664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac641b4-4a02-4a61-a70b-7990646d69a4 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:15.068180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-8ac641b4-4a02-4a61-a70b-7990646d69a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:15.144215 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:15.206789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1268/2508] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:15.210069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac641b4-4a02-4a61-a70b-7990646d69a4 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:15.211620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcc678-3d87-4d2a-9c52-add813ca8acb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:15.218581 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3e053-1699-4c6c-b288-d3323038c492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:15.220133 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3e053-1699-4c6c-b288-d3323038c492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.220842 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3e053-1699-4c6c-b288-d3323038c492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.221751 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3e053-1699-4c6c-b288-d3323038c492 admin admin] PATH_INFO: `/v3/projects/db9a3a8e3237470d8a5d0f49d71a00b9` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.224507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3e053-1699-4c6c-b288-d3323038c492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db9a3a8e3237470d8a5d0f49d71a00b9)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:15.225833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcc678-3d87-4d2a-9c52-add813ca8acb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:15.228561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3e053-1699-4c6c-b288-d3323038c492 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:15.232761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575a018-5f4e-4e0f-a177-16efcfd31ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:15.233622 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575a018-5f4e-4e0f-a177-16efcfd31ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.234126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575a018-5f4e-4e0f-a177-16efcfd31ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.234408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575a018-5f4e-4e0f-a177-16efcfd31ac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.235209 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575a018-5f4e-4e0f-a177-16efcfd31ac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:15.237326 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575a018-5f4e-4e0f-a177-16efcfd31ac7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:15.291105 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1269/2509] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:15 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:15.298443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575a018-5f4e-4e0f-a177-16efcfd31ac7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:15.333783 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6bc3e053-1699-4c6c-b288-d3323038c492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db9a3a8e3237470d8a5d0f49d71a00b9', 'request_id': 'req-6bc3e053-1699-4c6c-b288-d3323038c492'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:15.356056 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6bc3e053-1699-4c6c-b288-d3323038c492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db9a3a8e3237470d8a5d0f49d71a00b9', 'request_id': 'req-6bc3e053-1699-4c6c-b288-d3323038c492'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:15.373584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6bc3e053-1699-4c6c-b288-d3323038c492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db9a3a8e3237470d8a5d0f49d71a00b9', 'request_id': 'req-6bc3e053-1699-4c6c-b288-d3323038c492'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:15.401088 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6bc3e053-1699-4c6c-b288-d3323038c492 admin admin] The token cache is being invalidate because project db9a3a8e3237470d8a5d0f49d71a00b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:15.401872 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6bc3e053-1699-4c6c-b288-d3323038c492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:15.403605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6bc3e053-1699-4c6c-b288-d3323038c492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:15.406590 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1241/2510] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:15 2022] DELETE /identity/v3/projects/db9a3a8e3237470d8a5d0f49d71a00b9 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:15.412315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd961365-67ac-4d17-bb3e-73cf6794d993 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.412882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd961365-67ac-4d17-bb3e-73cf6794d993 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.413340 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd961365-67ac-4d17-bb3e-73cf6794d993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.450953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575a018-5f4e-4e0f-a177-16efcfd31ac7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:15.456914 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-dd961365-67ac-4d17-bb3e-73cf6794d993 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:15.459963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-dd961365-67ac-4d17-bb3e-73cf6794d993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:15.572133 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d49a7-90b2-4ea7-bf74-027cdbd3973c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:15.572927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d49a7-90b2-4ea7-bf74-027cdbd3973c admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.573352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d49a7-90b2-4ea7-bf74-027cdbd3973c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.573754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d49a7-90b2-4ea7-bf74-027cdbd3973c admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.574700 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d49a7-90b2-4ea7-bf74-027cdbd3973c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:15.577940 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d49a7-90b2-4ea7-bf74-027cdbd3973c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:15.583401 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1242/2511] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:15.588929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415f13a-9cae-4944-a05d-3d789363ba76 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.588929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415f13a-9cae-4944-a05d-3d789363ba76 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.588929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415f13a-9cae-4944-a05d-3d789363ba76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.626297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-b415f13a-9cae-4944-a05d-3d789363ba76 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:15.628204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b415f13a-9cae-4944-a05d-3d789363ba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:15.634503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1270/2512] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:15 2022] POST /identity/v3/users => generated 297 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:15.640361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d49a7-90b2-4ea7-bf74-027cdbd3973c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:15.642976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cd9d49a7-90b2-4ea7-bf74-027cdbd3973c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:15.648057 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1243/2513] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:15.654864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415f13a-9cae-4944-a05d-3d789363ba76 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:15.668099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d49a7-90b2-4ea7-bf74-027cdbd3973c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:15.674130 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415f13a-9cae-4944-a05d-3d789363ba76 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:15.677578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad60fb-de64-4fe2-b046-ac8a64a35882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:15.678715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad60fb-de64-4fe2-b046-ac8a64a35882 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.679295 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad60fb-de64-4fe2-b046-ac8a64a35882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.679882 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad60fb-de64-4fe2-b046-ac8a64a35882 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.684329 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb53d7c-1067-4ab6-9ea0-906391c01b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:15.685036 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad60fb-de64-4fe2-b046-ac8a64a35882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:15.685581 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb53d7c-1067-4ab6-9ea0-906391c01b15 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.686036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb53d7c-1067-4ab6-9ea0-906391c01b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.686210 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb53d7c-1067-4ab6-9ea0-906391c01b15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.686858 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb53d7c-1067-4ab6-9ea0-906391c01b15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:15.687204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb53d7c-1067-4ab6-9ea0-906391c01b15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:15.687394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad60fb-de64-4fe2-b046-ac8a64a35882 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:15.689752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb53d7c-1067-4ab6-9ea0-906391c01b15 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:15.710856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1271/2514] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:15.713108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad60fb-de64-4fe2-b046-ac8a64a35882 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:15.715339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1244/2515] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:15.731932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb53d7c-1067-4ab6-9ea0-906391c01b15 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:15.732208 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:15.732496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad60fb-de64-4fe2-b046-ac8a64a35882 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:15.740268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb53d7c-1067-4ab6-9ea0-906391c01b15 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:15.742022 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9b1b7-955b-40b5-bdf8-056b29d4d722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:15.743072 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9b1b7-955b-40b5-bdf8-056b29d4d722 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.745014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9b1b7-955b-40b5-bdf8-056b29d4d722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.745549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9b1b7-955b-40b5-bdf8-056b29d4d722 admin admin] PATH_INFO: `/v3/projects/953a948e68c3456285ebb339150c1bdd/users/a3b75c736ed94ef2a01a2935ff6b0c3c/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.747105 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802990a9-9b01-4ad4-bd7c-43fb52904012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:15.747944 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802990a9-9b01-4ad4-bd7c-43fb52904012 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.748351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802990a9-9b01-4ad4-bd7c-43fb52904012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.754375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802990a9-9b01-4ad4-bd7c-43fb52904012 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.754375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802990a9-9b01-4ad4-bd7c-43fb52904012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:15.754375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802990a9-9b01-4ad4-bd7c-43fb52904012 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:15.782500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9b1b7-955b-40b5-bdf8-056b29d4d722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=953a948e68c3456285ebb339150c1bdd, user_id=a3b75c736ed94ef2a01a2935ff6b0c3c, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:15.788106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9b1b7-955b-40b5-bdf8-056b29d4d722 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:15.808444 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1245/2516] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:15 2022] POST /identity/v3/projects => generated 371 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:15.813269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802990a9-9b01-4ad4-bd7c-43fb52904012 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:15.822347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1272/2517] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:15 2022] PUT /identity/v3/projects/953a948e68c3456285ebb339150c1bdd/users/a3b75c736ed94ef2a01a2935ff6b0c3c/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:15.830677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9b1b7-955b-40b5-bdf8-056b29d4d722 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:15.848288 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:15.919257 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802990a9-9b01-4ad4-bd7c-43fb52904012 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:15.926298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e26f2d-9110-4812-a476-4d5e99dd85e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:15.927505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e26f2d-9110-4812-a476-4d5e99dd85e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.927932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e26f2d-9110-4812-a476-4d5e99dd85e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.928459 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e26f2d-9110-4812-a476-4d5e99dd85e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.929549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e26f2d-9110-4812-a476-4d5e99dd85e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:15.932490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e26f2d-9110-4812-a476-4d5e99dd85e5 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:15.944736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9b1b7-955b-40b5-bdf8-056b29d4d722 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:15.954519 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7815a-376d-4f9a-aaba-0f86e90aeb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:15.955870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7815a-376d-4f9a-aaba-0f86e90aeb91 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:15.956962 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7815a-376d-4f9a-aaba-0f86e90aeb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:15.957614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7815a-376d-4f9a-aaba-0f86e90aeb91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:15.958927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7815a-376d-4f9a-aaba-0f86e90aeb91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:15.961260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7815a-376d-4f9a-aaba-0f86e90aeb91 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:15.968305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1246/2518] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:15 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:15.973940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e26f2d-9110-4812-a476-4d5e99dd85e5 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:15.982256 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1273/2519] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:15.984491 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7815a-376d-4f9a-aaba-0f86e90aeb91 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:15.995443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e26f2d-9110-4812-a476-4d5e99dd85e5 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:15.997230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7815a-376d-4f9a-aaba-0f86e90aeb91 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:16.001255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c034375-6180-44e6-9766-51540cbbcacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:16.002204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c034375-6180-44e6-9766-51540cbbcacf admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.002631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c034375-6180-44e6-9766-51540cbbcacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.003069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c034375-6180-44e6-9766-51540cbbcacf admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.004108 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c034375-6180-44e6-9766-51540cbbcacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:16.004429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2acfad-e88a-40ba-af95-5078feaa2336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:16.006690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c034375-6180-44e6-9766-51540cbbcacf admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:16.007297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2acfad-e88a-40ba-af95-5078feaa2336 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.008390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2acfad-e88a-40ba-af95-5078feaa2336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.008699 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2acfad-e88a-40ba-af95-5078feaa2336 admin admin] PATH_INFO: `/v3/projects/953a948e68c3456285ebb339150c1bdd/users/a3b75c736ed94ef2a01a2935ff6b0c3c/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.011444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2acfad-e88a-40ba-af95-5078feaa2336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=953a948e68c3456285ebb339150c1bdd, user_id=a3b75c736ed94ef2a01a2935ff6b0c3c, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:16.014172 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2acfad-e88a-40ba-af95-5078feaa2336 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:16.019218 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1247/2520] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:16.024032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c034375-6180-44e6-9766-51540cbbcacf admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:16.031105 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1274/2521] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:15 2022] PUT /identity/v3/projects/953a948e68c3456285ebb339150c1bdd/users/a3b75c736ed94ef2a01a2935ff6b0c3c/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:16.035856 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c034375-6180-44e6-9766-51540cbbcacf admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:16.042743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e821c-b298-44fd-bcdf-f9e98472a61e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.043052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e821c-b298-44fd-bcdf-f9e98472a61e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.043308 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e821c-b298-44fd-bcdf-f9e98472a61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.083994 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-949e821c-b298-44fd-bcdf-f9e98472a61e None None] MFA Rules not processed for user `a3b75c736ed94ef2a01a2935ff6b0c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:16.085248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-949e821c-b298-44fd-bcdf-f9e98472a61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:16.132402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf5820-e7f4-4e5a-8773-b1fe1f9db55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:16.133268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf5820-e7f4-4e5a-8773-b1fe1f9db55e admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.133557 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf5820-e7f4-4e5a-8773-b1fe1f9db55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.133822 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf5820-e7f4-4e5a-8773-b1fe1f9db55e admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.135385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf5820-e7f4-4e5a-8773-b1fe1f9db55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:16.137792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf5820-e7f4-4e5a-8773-b1fe1f9db55e admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:16.156062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1248/2522] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:16.164697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf5820-e7f4-4e5a-8773-b1fe1f9db55e admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:16.178960 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf5820-e7f4-4e5a-8773-b1fe1f9db55e admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:16.193360 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c002558-5c4c-46f6-8c8f-23fe7fd27c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:16.195623 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c002558-5c4c-46f6-8c8f-23fe7fd27c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.195978 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c002558-5c4c-46f6-8c8f-23fe7fd27c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.196328 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c002558-5c4c-46f6-8c8f-23fe7fd27c33 admin admin] PATH_INFO: `/v3/projects/ffbf1587d4ca4f379ef3c89a47cc42cd/users/542afb2e8b074fbd873969988fc3d849/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.217756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c002558-5c4c-46f6-8c8f-23fe7fd27c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffbf1587d4ca4f379ef3c89a47cc42cd, user_id=542afb2e8b074fbd873969988fc3d849, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:16.220733 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c002558-5c4c-46f6-8c8f-23fe7fd27c33 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:16.245943 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1249/2523] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:16 2022] PUT /identity/v3/projects/ffbf1587d4ca4f379ef3c89a47cc42cd/users/542afb2e8b074fbd873969988fc3d849/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:16.265004 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c2092-7467-40f1-9c5f-4cbfa9102926 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.265636 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c2092-7467-40f1-9c5f-4cbfa9102926 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.266082 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c2092-7467-40f1-9c5f-4cbfa9102926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.270559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1275/2524] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:16 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:16.288396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e821c-b298-44fd-bcdf-f9e98472a61e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:16.290048 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-949e821c-b298-44fd-bcdf-f9e98472a61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:16.291598 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-af7c2092-7467-40f1-9c5f-4cbfa9102926 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:16.292851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-af7c2092-7467-40f1-9c5f-4cbfa9102926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:16.412741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:16.422502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e821c-b298-44fd-bcdf-f9e98472a61e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:16.449366 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1250/2525] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:16.468744 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c2092-7467-40f1-9c5f-4cbfa9102926 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:16.475483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-af7c2092-7467-40f1-9c5f-4cbfa9102926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:16.499168 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:16.535704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f59db-92fb-4aab-b941-093863102141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:16.536451 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f59db-92fb-4aab-b941-093863102141 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.536730 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f59db-92fb-4aab-b941-093863102141 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.537000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f59db-92fb-4aab-b941-093863102141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.593814 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c2092-7467-40f1-9c5f-4cbfa9102926 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:16.597859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596b3f0-5461-4b59-a212-95b2ddd555ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:16.599384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596b3f0-5461-4b59-a212-95b2ddd555ac service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:16.599384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596b3f0-5461-4b59-a212-95b2ddd555ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:16.600730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596b3f0-5461-4b59-a212-95b2ddd555ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:16.623972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596b3f0-5461-4b59-a212-95b2ddd555ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:16.623972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596b3f0-5461-4b59-a212-95b2ddd555ac service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:16.631821 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f59db-92fb-4aab-b941-093863102141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:16.633939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f59db-92fb-4aab-b941-093863102141 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:16.634986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1251/2526] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:16.646575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1276/2527] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:16 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:17.312812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596b3f0-5461-4b59-a212-95b2ddd555ac service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.336984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596b3f0-5461-4b59-a212-95b2ddd555ac service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.342890 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b629d79-7411-43e7-a1c2-ac1049807493 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b75c736ed94ef2a01a2935ff6b0c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '953a948e68c3456285ebb339150c1bdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.343692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629d79-7411-43e7-a1c2-ac1049807493 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.344087 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629d79-7411-43e7-a1c2-ac1049807493 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.344455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629d79-7411-43e7-a1c2-ac1049807493 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.345263 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b629d79-7411-43e7-a1c2-ac1049807493 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:17.345655 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b629d79-7411-43e7-a1c2-ac1049807493 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.350246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b629d79-7411-43e7-a1c2-ac1049807493 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.385930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1252/2528] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:17.390327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f59db-92fb-4aab-b941-093863102141 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.401865 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f59db-92fb-4aab-b941-093863102141 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.409323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c24c06-a790-46d0-852f-821832e59966 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b75c736ed94ef2a01a2935ff6b0c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '953a948e68c3456285ebb339150c1bdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.410504 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c24c06-a790-46d0-852f-821832e59966 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.411068 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c24c06-a790-46d0-852f-821832e59966 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.412068 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c24c06-a790-46d0-852f-821832e59966 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.413783 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c24c06-a790-46d0-852f-821832e59966 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.416649 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c24c06-a790-46d0-852f-821832e59966 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.494989 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1277/2529] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:17 2022] POST /identity/v3/projects => generated 354 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:17.497242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b629d79-7411-43e7-a1c2-ac1049807493 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.517298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b629d79-7411-43e7-a1c2-ac1049807493 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.659174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06648c23-d4eb-457b-8033-40204689751d tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b75c736ed94ef2a01a2935ff6b0c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '953a948e68c3456285ebb339150c1bdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.660782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06648c23-d4eb-457b-8033-40204689751d tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.661376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06648c23-d4eb-457b-8033-40204689751d tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.662263 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06648c23-d4eb-457b-8033-40204689751d tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.663921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06648c23-d4eb-457b-8033-40204689751d tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:17.665815 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06648c23-d4eb-457b-8033-40204689751d tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.669613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06648c23-d4eb-457b-8033-40204689751d tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.707863 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1253/2530] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:17.774013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c24c06-a790-46d0-852f-821832e59966 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:17.792221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c24c06-a790-46d0-852f-821832e59966 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:17.802692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62bb50e-8be7-40cf-a457-8ae043c9c892 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b75c736ed94ef2a01a2935ff6b0c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '953a948e68c3456285ebb339150c1bdd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:17.807480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bb50e-8be7-40cf-a457-8ae043c9c892 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:17.807999 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bb50e-8be7-40cf-a457-8ae043c9c892 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:17.809143 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62bb50e-8be7-40cf-a457-8ae043c9c892 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] PATH_INFO: `/v3/projects/6b7942defe6243a1a46887017d5437f3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:17.810120 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62bb50e-8be7-40cf-a457-8ae043c9c892 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6b7942defe6243a1a46887017d5437f3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:17.812062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62bb50e-8be7-40cf-a457-8ae043c9c892 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:17.907641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b62bb50e-8be7-40cf-a457-8ae043c9c892 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b7942defe6243a1a46887017d5437f3', 'request_id': 'req-b62bb50e-8be7-40cf-a457-8ae043c9c892'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:17.923551 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b62bb50e-8be7-40cf-a457-8ae043c9c892 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b7942defe6243a1a46887017d5437f3', 'request_id': 'req-b62bb50e-8be7-40cf-a457-8ae043c9c892'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:17.937761 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b62bb50e-8be7-40cf-a457-8ae043c9c892 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b7942defe6243a1a46887017d5437f3', 'request_id': 'req-b62bb50e-8be7-40cf-a457-8ae043c9c892'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:17.973493 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b62bb50e-8be7-40cf-a457-8ae043c9c892 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] The token cache is being invalidate because project 6b7942defe6243a1a46887017d5437f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:17.978461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b62bb50e-8be7-40cf-a457-8ae043c9c892 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:17.979520 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b62bb50e-8be7-40cf-a457-8ae043c9c892 tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:17.986082 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1278/2531] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:17 2022] DELETE /identity/v3/projects/6b7942defe6243a1a46887017d5437f3 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:18.365819 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06648c23-d4eb-457b-8033-40204689751d tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:18.369936 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-06648c23-d4eb-457b-8033-40204689751d tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:18.657435 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06648c23-d4eb-457b-8033-40204689751d tempest-VolumeQuotaClassesTest-1901211470 tempest-VolumeQuotaClassesTest-1901211470-project-admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:18.870768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:18.907096 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02bcab-6047-4fa2-bcdd-d4acee2d8d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:18.908380 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02bcab-6047-4fa2-bcdd-d4acee2d8d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:18.910139 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02bcab-6047-4fa2-bcdd-d4acee2d8d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:18.910775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02bcab-6047-4fa2-bcdd-d4acee2d8d54 admin admin] PATH_INFO: `/v3/users/590c66767782487682b601a7f07c8b65` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:18.914973 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02bcab-6047-4fa2-bcdd-d4acee2d8d54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=590c66767782487682b601a7f07c8b65)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:18.925541 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02bcab-6047-4fa2-bcdd-d4acee2d8d54 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:19.010803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4d02bcab-6047-4fa2-bcdd-d4acee2d8d54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '590c66767782487682b601a7f07c8b65'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.059431 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4d02bcab-6047-4fa2-bcdd-d4acee2d8d54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '590c66767782487682b601a7f07c8b65'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.083813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4d02bcab-6047-4fa2-bcdd-d4acee2d8d54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '590c66767782487682b601a7f07c8b65'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.107322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4d02bcab-6047-4fa2-bcdd-d4acee2d8d54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '590c66767782487682b601a7f07c8b65'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.109691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4d02bcab-6047-4fa2-bcdd-d4acee2d8d54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '590c66767782487682b601a7f07c8b65'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.135007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1254/2532] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:18 2022] DELETE /identity/v3/users/590c66767782487682b601a7f07c8b65 => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:19.145401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa14ab-0d16-4c7d-8ab4-86cc12b856e9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:19.146080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa14ab-0d16-4c7d-8ab4-86cc12b856e9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:19.146511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa14ab-0d16-4c7d-8ab4-86cc12b856e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:19.173139 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a411fe-cdae-49a4-8f18-e688e53648e4 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:19.173954 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a411fe-cdae-49a4-8f18-e688e53648e4 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:19.174931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a411fe-cdae-49a4-8f18-e688e53648e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:19.245208 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-54fa14ab-0d16-4c7d-8ab4-86cc12b856e9 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:19.247113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-54fa14ab-0d16-4c7d-8ab4-86cc12b856e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:19.248931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-47a411fe-cdae-49a4-8f18-e688e53648e4 None None] MFA Rules not processed for user `542afb2e8b074fbd873969988fc3d849`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:19.250382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-47a411fe-cdae-49a4-8f18-e688e53648e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:19.273363 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:19.503875 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1279/2533] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:19.514094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa14ab-0d16-4c7d-8ab4-86cc12b856e9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:19.518267 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-54fa14ab-0d16-4c7d-8ab4-86cc12b856e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:19.521278 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1255/2534] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:19.536850 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a411fe-cdae-49a4-8f18-e688e53648e4 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:19.539984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-47a411fe-cdae-49a4-8f18-e688e53648e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:19.591316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a411fe-cdae-49a4-8f18-e688e53648e4 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:19.600206 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea5277-0c3a-4d2c-afc1-9850a9a0d418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:19.601479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea5277-0c3a-4d2c-afc1-9850a9a0d418 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:19.602062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea5277-0c3a-4d2c-afc1-9850a9a0d418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:19.602779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea5277-0c3a-4d2c-afc1-9850a9a0d418 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:19.604078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea5277-0c3a-4d2c-afc1-9850a9a0d418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:19.607657 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea5277-0c3a-4d2c-afc1-9850a9a0d418 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:19.671495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1256/2535] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:19 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:19.675579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea5277-0c3a-4d2c-afc1-9850a9a0d418 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:19.736954 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa14ab-0d16-4c7d-8ab4-86cc12b856e9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:19.784956 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:19.877814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1b62c-1c28-47fa-b62c-c01a35c53ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:19.878711 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1b62c-1c28-47fa-b62c-c01a35c53ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:19.879230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1b62c-1c28-47fa-b62c-c01a35c53ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:19.879701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1b62c-1c28-47fa-b62c-c01a35c53ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:19.882116 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea5277-0c3a-4d2c-afc1-9850a9a0d418 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:19.892083 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1b62c-1c28-47fa-b62c-c01a35c53ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:19.893487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95406933-13f0-4ff3-a0ec-9ff561ddc247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:19.895013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1b62c-1c28-47fa-b62c-c01a35c53ad7 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:19.897318 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95406933-13f0-4ff3-a0ec-9ff561ddc247 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:19.898062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95406933-13f0-4ff3-a0ec-9ff561ddc247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:19.898807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95406933-13f0-4ff3-a0ec-9ff561ddc247 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:19.900486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95406933-13f0-4ff3-a0ec-9ff561ddc247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:19.907377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95406933-13f0-4ff3-a0ec-9ff561ddc247 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:19.911791 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1280/2536] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:19.945756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1257/2537] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:19 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:19.950563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1b62c-1c28-47fa-b62c-c01a35c53ad7 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:19.968008 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1b62c-1c28-47fa-b62c-c01a35c53ad7 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:19.975150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7944c9-25b7-400e-981d-6c09b347f332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:19.976111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7944c9-25b7-400e-981d-6c09b347f332 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:19.976623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7944c9-25b7-400e-981d-6c09b347f332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:19.977139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7944c9-25b7-400e-981d-6c09b347f332 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:19.978098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7944c9-25b7-400e-981d-6c09b347f332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:19.981161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7944c9-25b7-400e-981d-6c09b347f332 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:19.997095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1281/2538] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:20.003360 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95406933-13f0-4ff3-a0ec-9ff561ddc247 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:20.017109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95406933-13f0-4ff3-a0ec-9ff561ddc247 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:20.026416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656e1b0-1d2e-4ed5-8eca-0001855f919a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:20.028140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656e1b0-1d2e-4ed5-8eca-0001855f919a admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:20.028699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656e1b0-1d2e-4ed5-8eca-0001855f919a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:20.029479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656e1b0-1d2e-4ed5-8eca-0001855f919a admin admin] PATH_INFO: `/v3/projects/8a1ba30664044d5b8ea1782c12a529b2/users/51d75351afea431da240fa50df845a9e/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:20.060992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656e1b0-1d2e-4ed5-8eca-0001855f919a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1ba30664044d5b8ea1782c12a529b2, user_id=51d75351afea431da240fa50df845a9e, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:20.066626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656e1b0-1d2e-4ed5-8eca-0001855f919a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:20.117072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1258/2539] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:20 2022] PUT /identity/v3/projects/8a1ba30664044d5b8ea1782c12a529b2/users/51d75351afea431da240fa50df845a9e/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:20.122138 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7944c9-25b7-400e-981d-6c09b347f332 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:20.135126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7944c9-25b7-400e-981d-6c09b347f332 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:20.243570 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602824e-5fe2-4e4f-9426-ce0d392e5410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:20.244478 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602824e-5fe2-4e4f-9426-ce0d392e5410 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:20.244646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602824e-5fe2-4e4f-9426-ce0d392e5410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:20.244818 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602824e-5fe2-4e4f-9426-ce0d392e5410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:20.245805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602824e-5fe2-4e4f-9426-ce0d392e5410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:20.249580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602824e-5fe2-4e4f-9426-ce0d392e5410 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:20.261426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:20.276264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1282/2540] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:20.276802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656e1b0-1d2e-4ed5-8eca-0001855f919a admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:20.288586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656e1b0-1d2e-4ed5-8eca-0001855f919a admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:20.295214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482d9cd-3de2-4e1f-8ca2-e10ea354c2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:20.296283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482d9cd-3de2-4e1f-8ca2-e10ea354c2bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:20.296725 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482d9cd-3de2-4e1f-8ca2-e10ea354c2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:20.297180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482d9cd-3de2-4e1f-8ca2-e10ea354c2bf admin admin] PATH_INFO: `/v3/projects/8a1ba30664044d5b8ea1782c12a529b2/users/51d75351afea431da240fa50df845a9e/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:20.299550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482d9cd-3de2-4e1f-8ca2-e10ea354c2bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1ba30664044d5b8ea1782c12a529b2, user_id=51d75351afea431da240fa50df845a9e, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:20.302883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482d9cd-3de2-4e1f-8ca2-e10ea354c2bf admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:20.317701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1259/2541] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:20 2022] PUT /identity/v3/projects/8a1ba30664044d5b8ea1782c12a529b2/users/51d75351afea431da240fa50df845a9e/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:21.591555 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602824e-5fe2-4e4f-9426-ce0d392e5410 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:21.593871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9602824e-5fe2-4e4f-9426-ce0d392e5410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:21.624975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602824e-5fe2-4e4f-9426-ce0d392e5410 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:21.734143 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:21.734915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:21.735269 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:21.735566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66 admin admin] PATH_INFO: `/v3/projects/665b6efdb4794e5abf46ef796bbbc169` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:21.736902 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=665b6efdb4794e5abf46ef796bbbc169)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:21.738897 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:21.840888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '665b6efdb4794e5abf46ef796bbbc169', 'request_id': 'req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:21.859100 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '665b6efdb4794e5abf46ef796bbbc169', 'request_id': 'req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:21.875052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '665b6efdb4794e5abf46ef796bbbc169', 'request_id': 'req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:21.910412 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66 admin admin] The token cache is being invalidate because project 665b6efdb4794e5abf46ef796bbbc169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:21.910705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:21.911413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:21.913559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1283/2542] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:21 2022] DELETE /identity/v3/projects/665b6efdb4794e5abf46ef796bbbc169 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:21.918640 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482d9cd-3de2-4e1f-8ca2-e10ea354c2bf admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:21.922595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9482d9cd-3de2-4e1f-8ca2-e10ea354c2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:21.938111 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482d9cd-3de2-4e1f-8ca2-e10ea354c2bf admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:22.038791 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df0871-bc8c-4b58-8f92-71dee102fe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:22.039833 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df0871-bc8c-4b58-8f92-71dee102fe6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:22.040439 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df0871-bc8c-4b58-8f92-71dee102fe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:22.041143 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df0871-bc8c-4b58-8f92-71dee102fe6e admin admin] PATH_INFO: `/v3/users/a3b75c736ed94ef2a01a2935ff6b0c3c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:22.043191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df0871-bc8c-4b58-8f92-71dee102fe6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b75c736ed94ef2a01a2935ff6b0c3c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:22.046290 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df0871-bc8c-4b58-8f92-71dee102fe6e admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:22.137750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-82df0871-bc8c-4b58-8f92-71dee102fe6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b75c736ed94ef2a01a2935ff6b0c3c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:22.154963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-82df0871-bc8c-4b58-8f92-71dee102fe6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b75c736ed94ef2a01a2935ff6b0c3c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:22.170546 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-82df0871-bc8c-4b58-8f92-71dee102fe6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b75c736ed94ef2a01a2935ff6b0c3c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:22.186805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-82df0871-bc8c-4b58-8f92-71dee102fe6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b75c736ed94ef2a01a2935ff6b0c3c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:22.187712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-82df0871-bc8c-4b58-8f92-71dee102fe6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b75c736ed94ef2a01a2935ff6b0c3c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:22.190364 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:22.204633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1260/2543] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:21 2022] DELETE /identity/v3/users/a3b75c736ed94ef2a01a2935ff6b0c3c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:22.561259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:22.601230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfa66d-c495-4c0e-8968-c1241ffd6a66 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:22.690821 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee01f7d-df06-4bf0-aaf8-617892537368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:22.691843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee01f7d-df06-4bf0-aaf8-617892537368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:22.692158 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee01f7d-df06-4bf0-aaf8-617892537368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:22.693062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee01f7d-df06-4bf0-aaf8-617892537368 admin admin] PATH_INFO: `/v3/projects/953a948e68c3456285ebb339150c1bdd` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:22.694290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee01f7d-df06-4bf0-aaf8-617892537368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=953a948e68c3456285ebb339150c1bdd)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:22.697310 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee01f7d-df06-4bf0-aaf8-617892537368 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:22.753694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:22.798520 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cee01f7d-df06-4bf0-aaf8-617892537368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '953a948e68c3456285ebb339150c1bdd', 'request_id': 'req-cee01f7d-df06-4bf0-aaf8-617892537368'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:22.808011 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f34ed8-8d45-482d-9705-db07c97e6dab None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:22.808773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f34ed8-8d45-482d-9705-db07c97e6dab None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:22.810048 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f34ed8-8d45-482d-9705-db07c97e6dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:22.815153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cee01f7d-df06-4bf0-aaf8-617892537368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '953a948e68c3456285ebb339150c1bdd', 'request_id': 'req-cee01f7d-df06-4bf0-aaf8-617892537368'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:22.828324 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cee01f7d-df06-4bf0-aaf8-617892537368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '953a948e68c3456285ebb339150c1bdd', 'request_id': 'req-cee01f7d-df06-4bf0-aaf8-617892537368'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:22.865019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cee01f7d-df06-4bf0-aaf8-617892537368 admin admin] The token cache is being invalidate because project 953a948e68c3456285ebb339150c1bdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:22.865019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cee01f7d-df06-4bf0-aaf8-617892537368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:22.865741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cee01f7d-df06-4bf0-aaf8-617892537368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:22.872754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1284/2544] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:22 2022] DELETE /identity/v3/projects/953a948e68c3456285ebb339150c1bdd => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:22.877102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b99dd2-207c-4987-9ce6-37a0caed40a8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:22.877102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b99dd2-207c-4987-9ce6-37a0caed40a8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:22.877102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b99dd2-207c-4987-9ce6-37a0caed40a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:22.885358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-70f34ed8-8d45-482d-9705-db07c97e6dab None None] MFA Rules not processed for user `51d75351afea431da240fa50df845a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:22.887296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-70f34ed8-8d45-482d-9705-db07c97e6dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:22.918845 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-81b99dd2-207c-4987-9ce6-37a0caed40a8 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:22.919938 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-81b99dd2-207c-4987-9ce6-37a0caed40a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:23.125009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1285/2545] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:23.126721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976ec8c-65a0-4906-8ce0-11cb99b852c7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:23.127210 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976ec8c-65a0-4906-8ce0-11cb99b852c7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:23.127564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976ec8c-65a0-4906-8ce0-11cb99b852c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:23.134911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1261/2546] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:23.141077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f34ed8-8d45-482d-9705-db07c97e6dab None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:23.143953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-70f34ed8-8d45-482d-9705-db07c97e6dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:23.173721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e976ec8c-65a0-4906-8ce0-11cb99b852c7 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:23.175413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e976ec8c-65a0-4906-8ce0-11cb99b852c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:23.190017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1286/2547] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:23.202936 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976ec8c-65a0-4906-8ce0-11cb99b852c7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:23.256363 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976ec8c-65a0-4906-8ce0-11cb99b852c7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:23.267669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0510a5-3f8c-4249-bb96-ceab6b02e0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:23.269592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0510a5-3f8c-4249-bb96-ceab6b02e0cc admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:23.270718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0510a5-3f8c-4249-bb96-ceab6b02e0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:23.271511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0510a5-3f8c-4249-bb96-ceab6b02e0cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:23.272993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0510a5-3f8c-4249-bb96-ceab6b02e0cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:23.273778 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0510a5-3f8c-4249-bb96-ceab6b02e0cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:23.277896 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0510a5-3f8c-4249-bb96-ceab6b02e0cc admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:23.339343 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f34ed8-8d45-482d-9705-db07c97e6dab None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:23.342617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1287/2548] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:23.347703 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9be204-fb73-4b67-9bf0-8a0857c2f4e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:23.348584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9be204-fb73-4b67-9bf0-8a0857c2f4e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:23.348938 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9be204-fb73-4b67-9bf0-8a0857c2f4e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:23.349273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9be204-fb73-4b67-9bf0-8a0857c2f4e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:23.352297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0510a5-3f8c-4249-bb96-ceab6b02e0cc admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:23.352585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ce9be204-fb73-4b67-9bf0-8a0857c2f4e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:23.369905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0510a5-3f8c-4249-bb96-ceab6b02e0cc admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:23.387319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40ca2c-f5d5-4efd-bc63-678a2a2cab20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:23.388734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40ca2c-f5d5-4efd-bc63-678a2a2cab20 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:23.389227 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40ca2c-f5d5-4efd-bc63-678a2a2cab20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:23.389724 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40ca2c-f5d5-4efd-bc63-678a2a2cab20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:23.391027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c40ca2c-f5d5-4efd-bc63-678a2a2cab20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:23.395667 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c40ca2c-f5d5-4efd-bc63-678a2a2cab20 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:23.470558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1288/2549] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:23 2022] POST /identity/v3/projects => generated 399 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:23.477455 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40ca2c-f5d5-4efd-bc63-678a2a2cab20 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:23.497055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9be204-fb73-4b67-9bf0-8a0857c2f4e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:23.498806 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9be204-fb73-4b67-9bf0-8a0857c2f4e1 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:23.510191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1262/2550] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:23.595741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40ca2c-f5d5-4efd-bc63-678a2a2cab20 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:23.617817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ba69a-a31b-4c50-bd2d-4ab0c415cba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:23.621437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ba69a-a31b-4c50-bd2d-4ab0c415cba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:23.621437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ba69a-a31b-4c50-bd2d-4ab0c415cba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:23.621993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ba69a-a31b-4c50-bd2d-4ab0c415cba6 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:23.623674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ba69a-a31b-4c50-bd2d-4ab0c415cba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:23.627694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ba69a-a31b-4c50-bd2d-4ab0c415cba6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:23.659763 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1289/2551] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:23 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:23.665636 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9be204-fb73-4b67-9bf0-8a0857c2f4e1 service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:23.684276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9be204-fb73-4b67-9bf0-8a0857c2f4e1 service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:23.700311 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4889af9-b3a7-4e6b-af94-055ddcada589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:23.701268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4889af9-b3a7-4e6b-af94-055ddcada589 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:23.701477 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4889af9-b3a7-4e6b-af94-055ddcada589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:23.701687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4889af9-b3a7-4e6b-af94-055ddcada589 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:23.702374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4889af9-b3a7-4e6b-af94-055ddcada589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:23.704266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4889af9-b3a7-4e6b-af94-055ddcada589 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:23.719545 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1263/2552] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:23.725394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ba69a-a31b-4c50-bd2d-4ab0c415cba6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:23.744502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ba69a-a31b-4c50-bd2d-4ab0c415cba6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:23.762821 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ec968-158e-40f1-a06e-8e0f172a8663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:23.763832 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ec968-158e-40f1-a06e-8e0f172a8663 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:23.765080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ec968-158e-40f1-a06e-8e0f172a8663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:23.765307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ec968-158e-40f1-a06e-8e0f172a8663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:23.766368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ec968-158e-40f1-a06e-8e0f172a8663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:23.768694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ec968-158e-40f1-a06e-8e0f172a8663 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:23.771751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:23.782679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1290/2553] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:23.788435 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4889af9-b3a7-4e6b-af94-055ddcada589 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:23.800801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4889af9-b3a7-4e6b-af94-055ddcada589 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:23.806678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72a49f-a5d1-4a92-9e12-8e03f3d29181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:23.809016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72a49f-a5d1-4a92-9e12-8e03f3d29181 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:23.809016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72a49f-a5d1-4a92-9e12-8e03f3d29181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:23.809016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72a49f-a5d1-4a92-9e12-8e03f3d29181 admin admin] PATH_INFO: `/v3/projects/9a3db59348904ab1bf6955b03cf3b19d/users/fcbc48d1301f40d8b2bdcc0ec01b7b6a/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:23.830643 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72a49f-a5d1-4a92-9e12-8e03f3d29181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a3db59348904ab1bf6955b03cf3b19d, user_id=fcbc48d1301f40d8b2bdcc0ec01b7b6a, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:23.835859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72a49f-a5d1-4a92-9e12-8e03f3d29181 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:23.855155 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1264/2554] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:23 2022] PUT /identity/v3/projects/9a3db59348904ab1bf6955b03cf3b19d/users/fcbc48d1301f40d8b2bdcc0ec01b7b6a/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:23.870416 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3e26c-0709-46b2-bf77-a10104e8acb6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:23.871111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3e26c-0709-46b2-bf77-a10104e8acb6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:23.871588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3e26c-0709-46b2-bf77-a10104e8acb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:23.943923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-39c3e26c-0709-46b2-bf77-a10104e8acb6 None None] MFA Rules not processed for user `fcbc48d1301f40d8b2bdcc0ec01b7b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:23.957171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-39c3e26c-0709-46b2-bf77-a10104e8acb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:24.112720 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1291/2555] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:23 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:24.117357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72a49f-a5d1-4a92-9e12-8e03f3d29181 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:24.127452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:24.218030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72a49f-a5d1-4a92-9e12-8e03f3d29181 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:24.223255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264aa87-b55e-4081-a4d1-8fd171ed091d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:24.223863 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264aa87-b55e-4081-a4d1-8fd171ed091d admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:24.224241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264aa87-b55e-4081-a4d1-8fd171ed091d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:24.224579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264aa87-b55e-4081-a4d1-8fd171ed091d admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:24.225567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264aa87-b55e-4081-a4d1-8fd171ed091d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:24.228076 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264aa87-b55e-4081-a4d1-8fd171ed091d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:24.278315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1265/2556] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:24 2022] POST /identity/v3/projects => generated 399 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:24.283324 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3e26c-0709-46b2-bf77-a10104e8acb6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:24.302388 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3e26c-0709-46b2-bf77-a10104e8acb6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:24.405802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4354f15-5bfb-44e9-926f-7db1643c1f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:24.407331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4354f15-5bfb-44e9-926f-7db1643c1f1e admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:24.407569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4354f15-5bfb-44e9-926f-7db1643c1f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:24.408163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4354f15-5bfb-44e9-926f-7db1643c1f1e admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:24.409322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4354f15-5bfb-44e9-926f-7db1643c1f1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:24.413181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4354f15-5bfb-44e9-926f-7db1643c1f1e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:24.457255 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1292/2557] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:24 2022] POST /identity/v3/users => generated 306 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:24.464336 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264aa87-b55e-4081-a4d1-8fd171ed091d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:24.476982 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264aa87-b55e-4081-a4d1-8fd171ed091d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:24.483395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474959db-17f0-472d-b451-a7079c1dd29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:24.484649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474959db-17f0-472d-b451-a7079c1dd29c admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:24.484649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474959db-17f0-472d-b451-a7079c1dd29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:24.484942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474959db-17f0-472d-b451-a7079c1dd29c admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:24.485671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474959db-17f0-472d-b451-a7079c1dd29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:24.488241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474959db-17f0-472d-b451-a7079c1dd29c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:24.517512 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4354f15-5bfb-44e9-926f-7db1643c1f1e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:24.526518 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1266/2558] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:24.539322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4354f15-5bfb-44e9-926f-7db1643c1f1e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:24.549259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f3f95-495e-4dd8-a6ee-f937ccb03ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:24.551241 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f3f95-495e-4dd8-a6ee-f937ccb03ba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:24.551684 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f3f95-495e-4dd8-a6ee-f937ccb03ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:24.552159 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f3f95-495e-4dd8-a6ee-f937ccb03ba1 admin admin] PATH_INFO: `/v3/projects/e5211f546f524e5789a9cce1169b4cc3/users/48b2ce833d3b4c0e8e8dec618dfaef33/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:24.604696 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f3f95-495e-4dd8-a6ee-f937ccb03ba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5211f546f524e5789a9cce1169b4cc3, user_id=48b2ce833d3b4c0e8e8dec618dfaef33, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:24.610673 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f3f95-495e-4dd8-a6ee-f937ccb03ba1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:24.643966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1293/2559] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:24 2022] PUT /identity/v3/projects/e5211f546f524e5789a9cce1169b4cc3/users/48b2ce833d3b4c0e8e8dec618dfaef33/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:24.649455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474959db-17f0-472d-b451-a7079c1dd29c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:24.661087 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474959db-17f0-472d-b451-a7079c1dd29c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:24.734793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f3f95-495e-4dd8-a6ee-f937ccb03ba1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:24.738486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-037f3f95-495e-4dd8-a6ee-f937ccb03ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:24.779692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde09f97-8e72-4489-8117-29972682699d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:24.785083 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde09f97-8e72-4489-8117-29972682699d admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:24.786700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde09f97-8e72-4489-8117-29972682699d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:24.787321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde09f97-8e72-4489-8117-29972682699d admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:24.788904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde09f97-8e72-4489-8117-29972682699d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:24.793336 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde09f97-8e72-4489-8117-29972682699d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:24.816624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1267/2560] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:24.824153 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde09f97-8e72-4489-8117-29972682699d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:24.842915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde09f97-8e72-4489-8117-29972682699d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:24.855603 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0ea2d-8f90-4c9a-bccf-143699c0a9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:24.857115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0ea2d-8f90-4c9a-bccf-143699c0a9a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:24.857731 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0ea2d-8f90-4c9a-bccf-143699c0a9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:24.858316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0ea2d-8f90-4c9a-bccf-143699c0a9a6 admin admin] PATH_INFO: `/v3/projects/e5211f546f524e5789a9cce1169b4cc3/users/48b2ce833d3b4c0e8e8dec618dfaef33/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:24.861972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0ea2d-8f90-4c9a-bccf-143699c0a9a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5211f546f524e5789a9cce1169b4cc3, user_id=48b2ce833d3b4c0e8e8dec618dfaef33, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:24.867832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0ea2d-8f90-4c9a-bccf-143699c0a9a6 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:24.887948 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1268/2561] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:24 2022] PUT /identity/v3/projects/e5211f546f524e5789a9cce1169b4cc3/users/48b2ce833d3b4c0e8e8dec618dfaef33/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:24.904583 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da660f47-af23-468d-ae07-bd9b8e694714 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:24.905405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da660f47-af23-468d-ae07-bd9b8e694714 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:24.907670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da660f47-af23-468d-ae07-bd9b8e694714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:24.910050 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f3f95-495e-4dd8-a6ee-f937ccb03ba1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:24.922898 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:24.972415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-da660f47-af23-468d-ae07-bd9b8e694714 None None] MFA Rules not processed for user `48b2ce833d3b4c0e8e8dec618dfaef33`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:24.974866 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-da660f47-af23-468d-ae07-bd9b8e694714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:25.047952 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a281a-fac0-417a-bbcb-2343fe066e76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:25.048738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a281a-fac0-417a-bbcb-2343fe066e76 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:25.049112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a281a-fac0-417a-bbcb-2343fe066e76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:25.049390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a281a-fac0-417a-bbcb-2343fe066e76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:25.051966 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-437a281a-fac0-417a-bbcb-2343fe066e76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:25.071207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:25.123562 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1269/2562] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:24 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:25.137390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da660f47-af23-468d-ae07-bd9b8e694714 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:25.139169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-da660f47-af23-468d-ae07-bd9b8e694714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:25.186810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a281a-fac0-417a-bbcb-2343fe066e76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:25.188855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a281a-fac0-417a-bbcb-2343fe066e76 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:25.215676 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1294/2563] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:25.250858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a281a-fac0-417a-bbcb-2343fe066e76 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:25.320279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da660f47-af23-468d-ae07-bd9b8e694714 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:25.329559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e077be-edfd-4636-8dbe-58d19236a913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:25.330868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e077be-edfd-4636-8dbe-58d19236a913 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:25.331489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e077be-edfd-4636-8dbe-58d19236a913 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:25.332138 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e077be-edfd-4636-8dbe-58d19236a913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:25.377631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e077be-edfd-4636-8dbe-58d19236a913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:25.378883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e077be-edfd-4636-8dbe-58d19236a913 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:25.397702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a281a-fac0-417a-bbcb-2343fe066e76 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:25.402712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1270/2564] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:25 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:25.405699 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5487b6-fcd0-419c-bd14-017fdb46170c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:25.406979 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5487b6-fcd0-419c-bd14-017fdb46170c service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:25.407842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5487b6-fcd0-419c-bd14-017fdb46170c service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:25.408410 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5487b6-fcd0-419c-bd14-017fdb46170c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:25.553137 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5487b6-fcd0-419c-bd14-017fdb46170c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:25.555881 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5487b6-fcd0-419c-bd14-017fdb46170c service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:25.566399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e077be-edfd-4636-8dbe-58d19236a913 service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:25.574571 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1295/2565] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:25.593063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e077be-edfd-4636-8dbe-58d19236a913 service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:25.600567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61e9a0-b54b-4c55-8e07-d79747b6d11e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:25.601803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61e9a0-b54b-4c55-8e07-d79747b6d11e service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:25.602629 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61e9a0-b54b-4c55-8e07-d79747b6d11e service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:25.603239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61e9a0-b54b-4c55-8e07-d79747b6d11e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:25.744251 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61e9a0-b54b-4c55-8e07-d79747b6d11e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:25.746902 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61e9a0-b54b-4c55-8e07-d79747b6d11e service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:25.764291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1271/2566] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:25 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:26.889409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5487b6-fcd0-419c-bd14-017fdb46170c service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:26.891733 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ee5487b6-fcd0-419c-bd14-017fdb46170c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:26.909036 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:26.988146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5487b6-fcd0-419c-bd14-017fdb46170c service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:26.995297 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f234d22-354e-46be-8b13-8d52f36883e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6281dcf2927a4ed2870125df8801be78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:26.996397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f234d22-354e-46be-8b13-8d52f36883e9 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:26.998638 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f234d22-354e-46be-8b13-8d52f36883e9 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:26.998638 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f234d22-354e-46be-8b13-8d52f36883e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:26.998893 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4f234d22-354e-46be-8b13-8d52f36883e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:27.006132 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61e9a0-b54b-4c55-8e07-d79747b6d11e service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:27.007608 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9c61e9a0-b54b-4c55-8e07-d79747b6d11e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:27.020299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f234d22-354e-46be-8b13-8d52f36883e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:27.022304 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f234d22-354e-46be-8b13-8d52f36883e9 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:27.040232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1296/2567] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:27.054591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61e9a0-b54b-4c55-8e07-d79747b6d11e service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:27.075954 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d938ea0-1efd-450e-9f19-43b010309497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:27.077036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d938ea0-1efd-450e-9f19-43b010309497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:27.077522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d938ea0-1efd-450e-9f19-43b010309497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:27.077969 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d938ea0-1efd-450e-9f19-43b010309497 admin admin] PATH_INFO: `/v3/users/da359f520a9d411eb52da5700c07f8d7` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:27.082140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d938ea0-1efd-450e-9f19-43b010309497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da359f520a9d411eb52da5700c07f8d7)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:27.086897 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d938ea0-1efd-450e-9f19-43b010309497 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:27.198832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3d938ea0-1efd-450e-9f19-43b010309497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da359f520a9d411eb52da5700c07f8d7'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:27.225249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3d938ea0-1efd-450e-9f19-43b010309497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da359f520a9d411eb52da5700c07f8d7'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:27.241377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3d938ea0-1efd-450e-9f19-43b010309497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da359f520a9d411eb52da5700c07f8d7'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:27.255286 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3d938ea0-1efd-450e-9f19-43b010309497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da359f520a9d411eb52da5700c07f8d7'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:27.260479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3d938ea0-1efd-450e-9f19-43b010309497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da359f520a9d411eb52da5700c07f8d7'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:27.263162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:27.283355 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1272/2568] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:27 2022] DELETE /identity/v3/users/da359f520a9d411eb52da5700c07f8d7 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:27.636486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f234d22-354e-46be-8b13-8d52f36883e9 service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:27.699882 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f234d22-354e-46be-8b13-8d52f36883e9 service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:27.877625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:27.878905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:27.879394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:27.879864 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] PATH_INFO: `/v3/projects/d5df463e3a1640ec8041d2b4bb6311f2` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:27.882124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5df463e3a1640ec8041d2b4bb6311f2)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:27.885961 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:27.977573 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5df463e3a1640ec8041d2b4bb6311f2', 'request_id': 'req-31a8bd52-abcf-472a-a167-bf05f58f38d7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:27.991190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5df463e3a1640ec8041d2b4bb6311f2', 'request_id': 'req-31a8bd52-abcf-472a-a167-bf05f58f38d7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:28.005869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5df463e3a1640ec8041d2b4bb6311f2', 'request_id': 'req-31a8bd52-abcf-472a-a167-bf05f58f38d7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:28.029214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] The token cache is being invalidate because project d5df463e3a1640ec8041d2b4bb6311f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:28.029703 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:28.030556 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:28.037640 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1297/2569] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:27 2022] DELETE /identity/v3/projects/d5df463e3a1640ec8041d2b4bb6311f2 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:28.039563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d938ea0-1efd-450e-9f19-43b010309497 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:28.042789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3d938ea0-1efd-450e-9f19-43b010309497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:28.059805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d938ea0-1efd-450e-9f19-43b010309497 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:28.179962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94529a-8505-4078-811b-1231690d7c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:28.181656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94529a-8505-4078-811b-1231690d7c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:28.182197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94529a-8505-4078-811b-1231690d7c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:28.182512 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94529a-8505-4078-811b-1231690d7c19 admin admin] PATH_INFO: `/v3/users/c4c610da9afb47cab01325b1dd637848` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:28.186903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94529a-8505-4078-811b-1231690d7c19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4c610da9afb47cab01325b1dd637848)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:28.189440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94529a-8505-4078-811b-1231690d7c19 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:28.281717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-dd94529a-8505-4078-811b-1231690d7c19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4c610da9afb47cab01325b1dd637848'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:28.308374 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-dd94529a-8505-4078-811b-1231690d7c19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4c610da9afb47cab01325b1dd637848'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:28.319804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-dd94529a-8505-4078-811b-1231690d7c19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4c610da9afb47cab01325b1dd637848'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:28.335800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-dd94529a-8505-4078-811b-1231690d7c19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4c610da9afb47cab01325b1dd637848'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:28.337118 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-dd94529a-8505-4078-811b-1231690d7c19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4c610da9afb47cab01325b1dd637848'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:28.345876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1273/2570] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:28 2022] DELETE /identity/v3/users/c4c610da9afb47cab01325b1dd637848 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:28.690952 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:28.694564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:28.733133 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8bd52-abcf-472a-a167-bf05f58f38d7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:28.803608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:28.901481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:28.903428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:28.903945 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:28.904371 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] PATH_INFO: `/v3/projects/ad365a4e010f4e1ca916cdd9b72e5065` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:28.905853 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad365a4e010f4e1ca916cdd9b72e5065)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:28.909051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:29.050384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad365a4e010f4e1ca916cdd9b72e5065', 'request_id': 'req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:29.072933 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad365a4e010f4e1ca916cdd9b72e5065', 'request_id': 'req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:29.091018 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad365a4e010f4e1ca916cdd9b72e5065', 'request_id': 'req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:29.129822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] The token cache is being invalidate because project ad365a4e010f4e1ca916cdd9b72e5065 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:29.130742 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:29.131809 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:29.147937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1298/2571] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:28 2022] DELETE /identity/v3/projects/ad365a4e010f4e1ca916cdd9b72e5065 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:31.183104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94529a-8505-4078-811b-1231690d7c19 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:31.186274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-dd94529a-8505-4078-811b-1231690d7c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:31.205980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94529a-8505-4078-811b-1231690d7c19 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:31.225754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:31.227533 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:31.321495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeacfe7-fb5d-4baa-84c3-5c6ab80010a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:31.322972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeacfe7-fb5d-4baa-84c3-5c6ab80010a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:31.323881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeacfe7-fb5d-4baa-84c3-5c6ab80010a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:31.324151 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed2696-bf3f-4b2d-87b6-5bc0eefb43bf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:31.324779 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeacfe7-fb5d-4baa-84c3-5c6ab80010a7 admin admin] PATH_INFO: `/v3/users/fcbc48d1301f40d8b2bdcc0ec01b7b6a` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:31.327159 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeacfe7-fb5d-4baa-84c3-5c6ab80010a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcbc48d1301f40d8b2bdcc0ec01b7b6a)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:31.329481 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeacfe7-fb5d-4baa-84c3-5c6ab80010a7 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:31.330261 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b9cd9-dcda-4610-a52c-2c2c8fe946f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:31.330928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b9cd9-dcda-4610-a52c-2c2c8fe946f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:31.331211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b9cd9-dcda-4610-a52c-2c2c8fe946f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:31.331566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b9cd9-dcda-4610-a52c-2c2c8fe946f6 admin admin] PATH_INFO: `/v3/users/849f875ede78424c81919df6d9c152dd` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:31.333378 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b9cd9-dcda-4610-a52c-2c2c8fe946f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=849f875ede78424c81919df6d9c152dd)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:31.336316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b9cd9-dcda-4610-a52c-2c2c8fe946f6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:31.356426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:31.409948 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-adeacfe7-fb5d-4baa-84c3-5c6ab80010a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcbc48d1301f40d8b2bdcc0ec01b7b6a'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.417963 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b72b9cd9-dcda-4610-a52c-2c2c8fe946f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '849f875ede78424c81919df6d9c152dd'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.424248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-adeacfe7-fb5d-4baa-84c3-5c6ab80010a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcbc48d1301f40d8b2bdcc0ec01b7b6a'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.433974 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b72b9cd9-dcda-4610-a52c-2c2c8fe946f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '849f875ede78424c81919df6d9c152dd'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.434825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b72b9cd9-dcda-4610-a52c-2c2c8fe946f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '849f875ede78424c81919df6d9c152dd'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.435851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-adeacfe7-fb5d-4baa-84c3-5c6ab80010a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcbc48d1301f40d8b2bdcc0ec01b7b6a'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.448271 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:31.451251 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b72b9cd9-dcda-4610-a52c-2c2c8fe946f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '849f875ede78424c81919df6d9c152dd'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.452191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-adeacfe7-fb5d-4baa-84c3-5c6ab80010a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcbc48d1301f40d8b2bdcc0ec01b7b6a'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.453301 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-adeacfe7-fb5d-4baa-84c3-5c6ab80010a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcbc48d1301f40d8b2bdcc0ec01b7b6a'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.462363 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b72b9cd9-dcda-4610-a52c-2c2c8fe946f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '849f875ede78424c81919df6d9c152dd'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.463215 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1274/2572] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:31 2022] DELETE /identity/v3/users/fcbc48d1301f40d8b2bdcc0ec01b7b6a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:31.468433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a52b0b-18e9-4f0b-affd-33c5fd2064d5 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:31.468717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a52b0b-18e9-4f0b-affd-33c5fd2064d5 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:31.469015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a52b0b-18e9-4f0b-affd-33c5fd2064d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:31.473952 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1299/2573] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:31 2022] DELETE /identity/v3/users/849f875ede78424c81919df6d9c152dd => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:31.496083 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-70a52b0b-18e9-4f0b-affd-33c5fd2064d5 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:31.497322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-70a52b0b-18e9-4f0b-affd-33c5fd2064d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:31.594397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1275/2574] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:31.607484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b9cd9-dcda-4610-a52c-2c2c8fe946f6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:31.609871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b72b9cd9-dcda-4610-a52c-2c2c8fe946f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:31.688233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a52b0b-18e9-4f0b-affd-33c5fd2064d5 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:31.689372 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-70a52b0b-18e9-4f0b-affd-33c5fd2064d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:31.714449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a52b0b-18e9-4f0b-affd-33c5fd2064d5 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:31.721797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9115e1-7c5f-4f04-a699-d52775c0d168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:31.722778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9115e1-7c5f-4f04-a699-d52775c0d168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:31.722948 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9115e1-7c5f-4f04-a699-d52775c0d168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:31.723291 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9115e1-7c5f-4f04-a699-d52775c0d168 admin admin] PATH_INFO: `/v3/projects/6470f32e4a954dcab3e0ccdd2736c1f6` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:31.723472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b9cd9-dcda-4610-a52c-2c2c8fe946f6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:31.724746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9115e1-7c5f-4f04-a699-d52775c0d168 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6470f32e4a954dcab3e0ccdd2736c1f6)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:31.727839 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255cdbd-25d6-450b-8b7a-0a50d0f40b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:31.728089 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9115e1-7c5f-4f04-a699-d52775c0d168 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:31.728691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255cdbd-25d6-450b-8b7a-0a50d0f40b7f service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:31.729082 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255cdbd-25d6-450b-8b7a-0a50d0f40b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:31.730576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255cdbd-25d6-450b-8b7a-0a50d0f40b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:31.746006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255cdbd-25d6-450b-8b7a-0a50d0f40b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:31.747610 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255cdbd-25d6-450b-8b7a-0a50d0f40b7f service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:31.772545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1300/2575] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:31.801643 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bd9115e1-7c5f-4f04-a699-d52775c0d168 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6470f32e4a954dcab3e0ccdd2736c1f6', 'request_id': 'req-bd9115e1-7c5f-4f04-a699-d52775c0d168'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.813415 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bd9115e1-7c5f-4f04-a699-d52775c0d168 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6470f32e4a954dcab3e0ccdd2736c1f6', 'request_id': 'req-bd9115e1-7c5f-4f04-a699-d52775c0d168'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.830382 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bd9115e1-7c5f-4f04-a699-d52775c0d168 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6470f32e4a954dcab3e0ccdd2736c1f6', 'request_id': 'req-bd9115e1-7c5f-4f04-a699-d52775c0d168'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.845550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bd9115e1-7c5f-4f04-a699-d52775c0d168 admin admin] The token cache is being invalidate because project 6470f32e4a954dcab3e0ccdd2736c1f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:31.845824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bd9115e1-7c5f-4f04-a699-d52775c0d168 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.846512 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bd9115e1-7c5f-4f04-a699-d52775c0d168 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:31.848475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1276/2576] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:31 2022] DELETE /identity/v3/projects/6470f32e4a954dcab3e0ccdd2736c1f6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:31.854112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b7c3f-bfc9-4a40-b687-7eb902804c4e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:31.854468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b7c3f-bfc9-4a40-b687-7eb902804c4e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:31.854662 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b7c3f-bfc9-4a40-b687-7eb902804c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:31.881960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ff6b7c3f-bfc9-4a40-b687-7eb902804c4e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:31.884464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ff6b7c3f-bfc9-4a40-b687-7eb902804c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:31.987649 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1301/2577] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:32.000361 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a640c-6402-4386-ae9e-9317393ee3b7 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.001026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a640c-6402-4386-ae9e-9317393ee3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.001874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a640c-6402-4386-ae9e-9317393ee3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.047289 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-031a640c-6402-4386-ae9e-9317393ee3b7 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:32.048876 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-031a640c-6402-4386-ae9e-9317393ee3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:32.058069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1277/2578] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:32.064817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b7c3f-bfc9-4a40-b687-7eb902804c4e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.078953 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b7c3f-bfc9-4a40-b687-7eb902804c4e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.085289 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961510d-4ee6-4e94-92e4-6caacd840601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.086273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961510d-4ee6-4e94-92e4-6caacd840601 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.086631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961510d-4ee6-4e94-92e4-6caacd840601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.087073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961510d-4ee6-4e94-92e4-6caacd840601 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.088156 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961510d-4ee6-4e94-92e4-6caacd840601 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:32.088663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961510d-4ee6-4e94-92e4-6caacd840601 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.091052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961510d-4ee6-4e94-92e4-6caacd840601 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.110932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1302/2579] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:32.117853 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a640c-6402-4386-ae9e-9317393ee3b7 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.131399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a640c-6402-4386-ae9e-9317393ee3b7 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.138467 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93ecec-8eca-465a-9e42-46c3b910e420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.139273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93ecec-8eca-465a-9e42-46c3b910e420 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.139648 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93ecec-8eca-465a-9e42-46c3b910e420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.140757 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93ecec-8eca-465a-9e42-46c3b910e420 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.140933 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93ecec-8eca-465a-9e42-46c3b910e420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.144060 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93ecec-8eca-465a-9e42-46c3b910e420 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.166044 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:32.198889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961510d-4ee6-4e94-92e4-6caacd840601 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.202700 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5961510d-4ee6-4e94-92e4-6caacd840601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:32.206949 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1278/2580] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:32 2022] POST /identity/v3/projects => generated 379 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:32.214645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93ecec-8eca-465a-9e42-46c3b910e420 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.226232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961510d-4ee6-4e94-92e4-6caacd840601 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.309830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.310847 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.311342 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.311802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] PATH_INFO: `/v3/projects/9a3db59348904ab1bf6955b03cf3b19d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.314431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a3db59348904ab1bf6955b03cf3b19d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.318356 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.336922 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:32.376009 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93ecec-8eca-465a-9e42-46c3b910e420 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.384169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341208c-9938-44e6-aaf8-2a16005a06bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.384905 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341208c-9938-44e6-aaf8-2a16005a06bd admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.385197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341208c-9938-44e6-aaf8-2a16005a06bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.385442 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341208c-9938-44e6-aaf8-2a16005a06bd admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.386085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341208c-9938-44e6-aaf8-2a16005a06bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.388062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341208c-9938-44e6-aaf8-2a16005a06bd admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.409556 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a3db59348904ab1bf6955b03cf3b19d', 'request_id': 'req-e45bebec-1100-4af5-8a40-ea5119e169d8'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.431531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a3db59348904ab1bf6955b03cf3b19d', 'request_id': 'req-e45bebec-1100-4af5-8a40-ea5119e169d8'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.439399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1279/2581] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:32 2022] POST /identity/v3/users => generated 290 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:32.444409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a3db59348904ab1bf6955b03cf3b19d', 'request_id': 'req-e45bebec-1100-4af5-8a40-ea5119e169d8'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.445536 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341208c-9938-44e6-aaf8-2a16005a06bd admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.463068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341208c-9938-44e6-aaf8-2a16005a06bd admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.468830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] The token cache is being invalidate because project 9a3db59348904ab1bf6955b03cf3b19d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:32.469021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.469530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.471460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1303/2582] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:32 2022] DELETE /identity/v3/projects/9a3db59348904ab1bf6955b03cf3b19d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:32.476595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.478623 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:32.564007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cbd3f-218a-4dab-9025-2919d94d1721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.564978 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cbd3f-218a-4dab-9025-2919d94d1721 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.565439 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cbd3f-218a-4dab-9025-2919d94d1721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.565894 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cbd3f-218a-4dab-9025-2919d94d1721 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.567087 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64cbd3f-218a-4dab-9025-2919d94d1721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.570302 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64cbd3f-218a-4dab-9025-2919d94d1721 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.584573 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bebec-1100-4af5-8a40-ea5119e169d8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.588532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1280/2583] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:32.592055 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5ad0b-9bee-4708-a982-30e81a61016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.592715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5ad0b-9bee-4708-a982-30e81a61016a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.593006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5ad0b-9bee-4708-a982-30e81a61016a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.593172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cbd3f-218a-4dab-9025-2919d94d1721 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.593283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5ad0b-9bee-4708-a982-30e81a61016a admin admin] PATH_INFO: `/v3/users/48b2ce833d3b4c0e8e8dec618dfaef33` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.594479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5ad0b-9bee-4708-a982-30e81a61016a admin admin] RBAC: Authorizing `identity:delete_user(user_id=48b2ce833d3b4c0e8e8dec618dfaef33)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.595357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e64cbd3f-218a-4dab-9025-2919d94d1721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:32.596618 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5ad0b-9bee-4708-a982-30e81a61016a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.616548 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cbd3f-218a-4dab-9025-2919d94d1721 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.625322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f8bca-85a1-47df-81ff-b741e0c8c0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.626351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f8bca-85a1-47df-81ff-b741e0c8c0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.626818 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f8bca-85a1-47df-81ff-b741e0c8c0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.627294 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f8bca-85a1-47df-81ff-b741e0c8c0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.628438 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f8bca-85a1-47df-81ff-b741e0c8c0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.631429 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f8bca-85a1-47df-81ff-b741e0c8c0c2 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.652735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1281/2584] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:32.659007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f8bca-85a1-47df-81ff-b741e0c8c0c2 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.665001 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-40c5ad0b-9bee-4708-a982-30e81a61016a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48b2ce833d3b4c0e8e8dec618dfaef33'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.671620 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f8bca-85a1-47df-81ff-b741e0c8c0c2 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:32.676850 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-40c5ad0b-9bee-4708-a982-30e81a61016a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48b2ce833d3b4c0e8e8dec618dfaef33'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.678059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-40c5ad0b-9bee-4708-a982-30e81a61016a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48b2ce833d3b4c0e8e8dec618dfaef33'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.684439 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc367a-9e63-4ca3-9af0-139cc13a892d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:32.685881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc367a-9e63-4ca3-9af0-139cc13a892d admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.686317 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc367a-9e63-4ca3-9af0-139cc13a892d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.686944 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc367a-9e63-4ca3-9af0-139cc13a892d admin admin] PATH_INFO: `/v3/projects/2166eb7e8d1344eda9d963b32a8b4f0d/users/11f63b2804304a5f8274dffd5e8a602d/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.697951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-40c5ad0b-9bee-4708-a982-30e81a61016a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48b2ce833d3b4c0e8e8dec618dfaef33'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.705780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-40c5ad0b-9bee-4708-a982-30e81a61016a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48b2ce833d3b4c0e8e8dec618dfaef33'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:32.713436 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1304/2585] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:32 2022] DELETE /identity/v3/users/48b2ce833d3b4c0e8e8dec618dfaef33 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:32.715617 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc367a-9e63-4ca3-9af0-139cc13a892d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2166eb7e8d1344eda9d963b32a8b4f0d, user_id=11f63b2804304a5f8274dffd5e8a602d, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:32.720810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc367a-9e63-4ca3-9af0-139cc13a892d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:32.745896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1282/2586] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:32 2022] PUT /identity/v3/projects/2166eb7e8d1344eda9d963b32a8b4f0d/users/11f63b2804304a5f8274dffd5e8a602d/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:32.759044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6575d-3876-4e05-92b4-53e3ebcb2e77 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:32.759523 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6575d-3876-4e05-92b4-53e3ebcb2e77 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:32.760490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6575d-3876-4e05-92b4-53e3ebcb2e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:32.810068 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b1a6575d-3876-4e05-92b4-53e3ebcb2e77 None None] MFA Rules not processed for user `11f63b2804304a5f8274dffd5e8a602d`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:32.812070 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b1a6575d-3876-4e05-92b4-53e3ebcb2e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:32.820717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:32.903361 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1305/2587] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:32.908370 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc367a-9e63-4ca3-9af0-139cc13a892d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:32.910294 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-03cc367a-9e63-4ca3-9af0-139cc13a892d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:32.986354 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:33.010495 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc367a-9e63-4ca3-9af0-139cc13a892d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:33.015937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c65b25-95c6-4b3a-b619-9bc434589451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:33.016558 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c65b25-95c6-4b3a-b619-9bc434589451 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.016832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c65b25-95c6-4b3a-b619-9bc434589451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.017102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c65b25-95c6-4b3a-b619-9bc434589451 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.017744 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c65b25-95c6-4b3a-b619-9bc434589451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:33.019687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c65b25-95c6-4b3a-b619-9bc434589451 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:33.053155 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1283/2588] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:32 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:33.060237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6575d-3876-4e05-92b4-53e3ebcb2e77 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.073882 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6575d-3876-4e05-92b4-53e3ebcb2e77 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:33.131273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c65b25-95c6-4b3a-b619-9bc434589451 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.136072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-89c65b25-95c6-4b3a-b619-9bc434589451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:33.181434 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8806099-e3bd-4326-ac36-f840592c99fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:33.182044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8806099-e3bd-4326-ac36-f840592c99fa admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.182563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8806099-e3bd-4326-ac36-f840592c99fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.182994 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8806099-e3bd-4326-ac36-f840592c99fa admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.184568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8806099-e3bd-4326-ac36-f840592c99fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:33.188184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8806099-e3bd-4326-ac36-f840592c99fa admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:33.231563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1306/2589] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:33 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:33.239218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8806099-e3bd-4326-ac36-f840592c99fa admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.257654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8806099-e3bd-4326-ac36-f840592c99fa admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:33.258636 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c65b25-95c6-4b3a-b619-9bc434589451 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:33.264580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343c40d-05ae-4f14-b117-daf151f68e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:33.265342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343c40d-05ae-4f14-b117-daf151f68e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.265570 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86367e5d-02b0-4015-b680-f1dee97242b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:33.265834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343c40d-05ae-4f14-b117-daf151f68e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.266120 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343c40d-05ae-4f14-b117-daf151f68e31 admin admin] PATH_INFO: `/v3/projects/e5211f546f524e5789a9cce1169b4cc3` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.266666 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86367e5d-02b0-4015-b680-f1dee97242b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.267338 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86367e5d-02b0-4015-b680-f1dee97242b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.267787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86367e5d-02b0-4015-b680-f1dee97242b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.268675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86367e5d-02b0-4015-b680-f1dee97242b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:33.268869 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343c40d-05ae-4f14-b117-daf151f68e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5211f546f524e5789a9cce1169b4cc3)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:33.270836 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343c40d-05ae-4f14-b117-daf151f68e31 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:33.271502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86367e5d-02b0-4015-b680-f1dee97242b0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:33.285997 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1307/2590] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:33.296042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86367e5d-02b0-4015-b680-f1dee97242b0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.319185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86367e5d-02b0-4015-b680-f1dee97242b0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:33.341452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5343c40d-05ae-4f14-b117-daf151f68e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5211f546f524e5789a9cce1169b4cc3', 'request_id': 'req-5343c40d-05ae-4f14-b117-daf151f68e31'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:33.351543 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5343c40d-05ae-4f14-b117-daf151f68e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5211f546f524e5789a9cce1169b4cc3', 'request_id': 'req-5343c40d-05ae-4f14-b117-daf151f68e31'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:33.360710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5343c40d-05ae-4f14-b117-daf151f68e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5211f546f524e5789a9cce1169b4cc3', 'request_id': 'req-5343c40d-05ae-4f14-b117-daf151f68e31'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:33.375245 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5343c40d-05ae-4f14-b117-daf151f68e31 admin admin] The token cache is being invalidate because project e5211f546f524e5789a9cce1169b4cc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:33.375677 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5343c40d-05ae-4f14-b117-daf151f68e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:33.376537 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-5343c40d-05ae-4f14-b117-daf151f68e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:33.378216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1284/2591] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:33 2022] DELETE /identity/v3/projects/e5211f546f524e5789a9cce1169b4cc3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:33.388667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bc4e0-7bf1-4c03-844c-1c3e0bd34167 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.389230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bc4e0-7bf1-4c03-844c-1c3e0bd34167 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.389701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bc4e0-7bf1-4c03-844c-1c3e0bd34167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.398125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29b3a0-54d6-43c6-a618-c67ca77c3651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:33.398816 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29b3a0-54d6-43c6-a618-c67ca77c3651 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.399203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29b3a0-54d6-43c6-a618-c67ca77c3651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.399661 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29b3a0-54d6-43c6-a618-c67ca77c3651 admin admin] PATH_INFO: `/v3/projects/2188bfa324394fb08f59a8e334d03302/users/9a4f0fc2b26441bcaa9ccb4fdc03685e/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.427162 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29b3a0-54d6-43c6-a618-c67ca77c3651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2188bfa324394fb08f59a8e334d03302, user_id=9a4f0fc2b26441bcaa9ccb4fdc03685e, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:33.432358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29b3a0-54d6-43c6-a618-c67ca77c3651 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:33.434378 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-d50bc4e0-7bf1-4c03-844c-1c3e0bd34167 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:33.436113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d50bc4e0-7bf1-4c03-844c-1c3e0bd34167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:33.450690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1285/2592] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:33.456005 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1308/2593] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:33 2022] PUT /identity/v3/projects/2188bfa324394fb08f59a8e334d03302/users/9a4f0fc2b26441bcaa9ccb4fdc03685e/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:33.459954 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044444e-7a1e-4915-89a5-faa09e415040 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.460243 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044444e-7a1e-4915-89a5-faa09e415040 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.460723 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044444e-7a1e-4915-89a5-faa09e415040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.462213 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29b3a0-54d6-43c6-a618-c67ca77c3651 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.464893 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bc29b3a0-54d6-43c6-a618-c67ca77c3651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:33.482587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:33.490937 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-c044444e-7a1e-4915-89a5-faa09e415040 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:33.492190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c044444e-7a1e-4915-89a5-faa09e415040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:33.551970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:33.576129 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1286/2594] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:33.582606 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044444e-7a1e-4915-89a5-faa09e415040 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.583058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29b3a0-54d6-43c6-a618-c67ca77c3651 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:33.594377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62b2c9-786d-4f19-96f3-ae7f3e952a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:33.594721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044444e-7a1e-4915-89a5-faa09e415040 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:33.595121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62b2c9-786d-4f19-96f3-ae7f3e952a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.595726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62b2c9-786d-4f19-96f3-ae7f3e952a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.596317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62b2c9-786d-4f19-96f3-ae7f3e952a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.597565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62b2c9-786d-4f19-96f3-ae7f3e952a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:33.600290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62b2c9-786d-4f19-96f3-ae7f3e952a22 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:33.600543 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20392a9-9ae8-44cb-851e-695f0fd336c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:33.601174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20392a9-9ae8-44cb-851e-695f0fd336c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.601577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20392a9-9ae8-44cb-851e-695f0fd336c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.601941 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20392a9-9ae8-44cb-851e-695f0fd336c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.603392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20392a9-9ae8-44cb-851e-695f0fd336c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:33.603830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20392a9-9ae8-44cb-851e-695f0fd336c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:33.605738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20392a9-9ae8-44cb-851e-695f0fd336c0 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:33.616386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1309/2595] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:33.620996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62b2c9-786d-4f19-96f3-ae7f3e952a22 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.628093 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1287/2596] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:43:33.633630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62b2c9-786d-4f19-96f3-ae7f3e952a22 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:33.634208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20392a9-9ae8-44cb-851e-695f0fd336c0 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.642590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99504c-1eb6-4889-b824-27cc558a8ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:33.644034 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99504c-1eb6-4889-b824-27cc558a8ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.644722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99504c-1eb6-4889-b824-27cc558a8ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.645464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99504c-1eb6-4889-b824-27cc558a8ad3 admin admin] PATH_INFO: `/v3/projects/2188bfa324394fb08f59a8e334d03302/users/9a4f0fc2b26441bcaa9ccb4fdc03685e/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.646108 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20392a9-9ae8-44cb-851e-695f0fd336c0 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:33.649042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99504c-1eb6-4889-b824-27cc558a8ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2188bfa324394fb08f59a8e334d03302, user_id=9a4f0fc2b26441bcaa9ccb4fdc03685e, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:33.651627 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17f623-6c3a-487f-b2c2-94450a13fa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:33.652330 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17f623-6c3a-487f-b2c2-94450a13fa25 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.652602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17f623-6c3a-487f-b2c2-94450a13fa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.652871 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17f623-6c3a-487f-b2c2-94450a13fa25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.653518 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17f623-6c3a-487f-b2c2-94450a13fa25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:33.655426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17f623-6c3a-487f-b2c2-94450a13fa25 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:33.657087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99504c-1eb6-4889-b824-27cc558a8ad3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:33.679980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1310/2597] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:33 2022] PUT /identity/v3/projects/2188bfa324394fb08f59a8e334d03302/users/9a4f0fc2b26441bcaa9ccb4fdc03685e/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:33.692865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1288/2598] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:33 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:33.694647 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c04b79-9d67-4226-8113-092d287f4e44 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.695124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c04b79-9d67-4226-8113-092d287f4e44 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.695621 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c04b79-9d67-4226-8113-092d287f4e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.699080 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17f623-6c3a-487f-b2c2-94450a13fa25 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.751421 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-50c04b79-9d67-4226-8113-092d287f4e44 None None] MFA Rules not processed for user `9a4f0fc2b26441bcaa9ccb4fdc03685e`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:33.754012 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-50c04b79-9d67-4226-8113-092d287f4e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:33.825865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17f623-6c3a-487f-b2c2-94450a13fa25 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:33.843554 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c5c9f-1ad6-4886-8f2d-0954fb9d9c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:33.844829 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c5c9f-1ad6-4886-8f2d-0954fb9d9c76 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.845420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c5c9f-1ad6-4886-8f2d-0954fb9d9c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.845984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c5c9f-1ad6-4886-8f2d-0954fb9d9c76 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.848024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c5c9f-1ad6-4886-8f2d-0954fb9d9c76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:33.850999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c5c9f-1ad6-4886-8f2d-0954fb9d9c76 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:33.883124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1289/2599] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:33 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:33.889009 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c5c9f-1ad6-4886-8f2d-0954fb9d9c76 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.893284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1311/2600] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:33.906984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c5c9f-1ad6-4886-8f2d-0954fb9d9c76 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:33.914198 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c04b79-9d67-4226-8113-092d287f4e44 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.914423 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841560a1-af2a-4ed5-8781-7eea02241e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:33.915209 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841560a1-af2a-4ed5-8781-7eea02241e71 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.915505 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841560a1-af2a-4ed5-8781-7eea02241e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.915851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841560a1-af2a-4ed5-8781-7eea02241e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.916446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841560a1-af2a-4ed5-8781-7eea02241e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:33.917231 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-50c04b79-9d67-4226-8113-092d287f4e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:33.918087 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841560a1-af2a-4ed5-8781-7eea02241e71 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:33.930140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1290/2601] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:33.935292 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841560a1-af2a-4ed5-8781-7eea02241e71 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.947827 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841560a1-af2a-4ed5-8781-7eea02241e71 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:33.957198 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a710fd-e326-4c40-bc98-81154e6a3984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:33.957926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a710fd-e326-4c40-bc98-81154e6a3984 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:33.958263 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a710fd-e326-4c40-bc98-81154e6a3984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:33.958592 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a710fd-e326-4c40-bc98-81154e6a3984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:33.959376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a710fd-e326-4c40-bc98-81154e6a3984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:33.961155 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a710fd-e326-4c40-bc98-81154e6a3984 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:33.977919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1291/2602] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:33.982588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a710fd-e326-4c40-bc98-81154e6a3984 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:33.996230 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a710fd-e326-4c40-bc98-81154e6a3984 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:34.002629 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d73b6-12be-483a-94f5-6ec94d9fe584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:34.003216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d73b6-12be-483a-94f5-6ec94d9fe584 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:34.003487 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d73b6-12be-483a-94f5-6ec94d9fe584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:34.003753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d73b6-12be-483a-94f5-6ec94d9fe584 admin admin] PATH_INFO: `/v3/projects/394486dea38b4fbe8d070fc198aa4c6c/users/0f44e04fc67b4ec4a0b013ab06e6bf0d/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:34.007387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c04b79-9d67-4226-8113-092d287f4e44 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:34.011401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc129e-a6bb-4674-8fa2-ba0d4001cec9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:34.012063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc129e-a6bb-4674-8fa2-ba0d4001cec9 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:34.012343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc129e-a6bb-4674-8fa2-ba0d4001cec9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:34.012612 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc129e-a6bb-4674-8fa2-ba0d4001cec9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:34.022956 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc129e-a6bb-4674-8fa2-ba0d4001cec9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:34.023407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d73b6-12be-483a-94f5-6ec94d9fe584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=394486dea38b4fbe8d070fc198aa4c6c, user_id=0f44e04fc67b4ec4a0b013ab06e6bf0d, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:34.024915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc129e-a6bb-4674-8fa2-ba0d4001cec9 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:34.026177 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d73b6-12be-483a-94f5-6ec94d9fe584 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:34.036742 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1312/2603] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:34.044796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1292/2604] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:33 2022] PUT /identity/v3/projects/394486dea38b4fbe8d070fc198aa4c6c/users/0f44e04fc67b4ec4a0b013ab06e6bf0d/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:34.064336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424177d-f236-4fdc-9faa-da5429aa5200 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:34.064824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424177d-f236-4fdc-9faa-da5429aa5200 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:34.065038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424177d-f236-4fdc-9faa-da5429aa5200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:34.103576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4424177d-f236-4fdc-9faa-da5429aa5200 None None] MFA Rules not processed for user `0f44e04fc67b4ec4a0b013ab06e6bf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:34.104928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4424177d-f236-4fdc-9faa-da5429aa5200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:34.118928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:34.144769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d73b6-12be-483a-94f5-6ec94d9fe584 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:34.202064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1313/2605] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:34.207506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424177d-f236-4fdc-9faa-da5429aa5200 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:34.227621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d73b6-12be-483a-94f5-6ec94d9fe584 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:34.233190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc55ee-840d-44e8-9f31-71d0a7694abf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:34.233847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc55ee-840d-44e8-9f31-71d0a7694abf service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:34.234250 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc55ee-840d-44e8-9f31-71d0a7694abf service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:34.234523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc55ee-840d-44e8-9f31-71d0a7694abf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:34.236709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d2bc55ee-840d-44e8-9f31-71d0a7694abf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:34.252539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:34.289504 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424177d-f236-4fdc-9faa-da5429aa5200 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:34.295786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d33e3a-9b04-4f44-9544-8a1916b0d6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:34.296511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d33e3a-9b04-4f44-9544-8a1916b0d6de admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:34.296858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d33e3a-9b04-4f44-9544-8a1916b0d6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:34.297223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d33e3a-9b04-4f44-9544-8a1916b0d6de admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:34.297912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d33e3a-9b04-4f44-9544-8a1916b0d6de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:34.299917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d33e3a-9b04-4f44-9544-8a1916b0d6de admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:34.335218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1314/2606] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:34 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:34.336226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc55ee-840d-44e8-9f31-71d0a7694abf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:34.338395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc55ee-840d-44e8-9f31-71d0a7694abf service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:34.340272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d33e3a-9b04-4f44-9544-8a1916b0d6de admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:34.349388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1293/2607] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:34.426167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d33e3a-9b04-4f44-9544-8a1916b0d6de admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:34.434410 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9740e5a-131b-4263-af83-e36f05117636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:34.435652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9740e5a-131b-4263-af83-e36f05117636 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:34.436033 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9740e5a-131b-4263-af83-e36f05117636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:34.436344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9740e5a-131b-4263-af83-e36f05117636 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:34.437143 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9740e5a-131b-4263-af83-e36f05117636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:34.439390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9740e5a-131b-4263-af83-e36f05117636 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:34.467463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1315/2608] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:34 2022] POST /identity/v3/users => generated 296 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:34.472023 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc55ee-840d-44e8-9f31-71d0a7694abf service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:34.487671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc55ee-840d-44e8-9f31-71d0a7694abf service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:34.494587 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04c305-eb7b-4c55-aeb4-03b0a26b5817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:34.495345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04c305-eb7b-4c55-aeb4-03b0a26b5817 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:34.495717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04c305-eb7b-4c55-aeb4-03b0a26b5817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:34.496196 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04c305-eb7b-4c55-aeb4-03b0a26b5817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:34.497483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04c305-eb7b-4c55-aeb4-03b0a26b5817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:34.500004 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04c305-eb7b-4c55-aeb4-03b0a26b5817 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:34.526398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1294/2609] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:34.528609 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9740e5a-131b-4263-af83-e36f05117636 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:34.544379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9740e5a-131b-4263-af83-e36f05117636 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:34.552160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6916353-d7c7-48ce-bd71-18efb6e498e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:34.552160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6916353-d7c7-48ce-bd71-18efb6e498e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:34.552160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6916353-d7c7-48ce-bd71-18efb6e498e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:34.552437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6916353-d7c7-48ce-bd71-18efb6e498e0 admin admin] PATH_INFO: `/v3/projects/1843694a25e74b58a409656e4a0750ab/users/b60f92ebcd344597adb384754595929f/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:34.573802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6916353-d7c7-48ce-bd71-18efb6e498e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1843694a25e74b58a409656e4a0750ab, user_id=b60f92ebcd344597adb384754595929f, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:34.577927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6916353-d7c7-48ce-bd71-18efb6e498e0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:34.590306 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:34.598426 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1316/2610] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:34 2022] PUT /identity/v3/projects/1843694a25e74b58a409656e4a0750ab/users/b60f92ebcd344597adb384754595929f/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:34.605615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04c305-eb7b-4c55-aeb4-03b0a26b5817 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:34.617966 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04c305-eb7b-4c55-aeb4-03b0a26b5817 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:34.683862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94b380-7c20-4637-bba2-81510ede9519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:34.684262 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94b380-7c20-4637-bba2-81510ede9519 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:34.684549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94b380-7c20-4637-bba2-81510ede9519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:34.684867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94b380-7c20-4637-bba2-81510ede9519 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:34.685637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94b380-7c20-4637-bba2-81510ede9519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:34.687275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94b380-7c20-4637-bba2-81510ede9519 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:34.699252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1295/2611] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:34.703935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6916353-d7c7-48ce-bd71-18efb6e498e0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:34.722746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6916353-d7c7-48ce-bd71-18efb6e498e0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:34.728899 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b5b63-c93a-47e2-b2e5-537b17f763f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:34.729596 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b5b63-c93a-47e2-b2e5-537b17f763f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:34.729888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b5b63-c93a-47e2-b2e5-537b17f763f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:34.730159 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b5b63-c93a-47e2-b2e5-537b17f763f0 admin admin] PATH_INFO: `/v3/projects/1843694a25e74b58a409656e4a0750ab/users/b60f92ebcd344597adb384754595929f/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:34.732255 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b5b63-c93a-47e2-b2e5-537b17f763f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1843694a25e74b58a409656e4a0750ab, user_id=b60f92ebcd344597adb384754595929f, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:34.734938 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b5b63-c93a-47e2-b2e5-537b17f763f0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:34.747126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1317/2612] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:34 2022] PUT /identity/v3/projects/1843694a25e74b58a409656e4a0750ab/users/b60f92ebcd344597adb384754595929f/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:34.766157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fda719-ea41-4f4c-b74c-798dda5607b0 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:34.766719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fda719-ea41-4f4c-b74c-798dda5607b0 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:34.766995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fda719-ea41-4f4c-b74c-798dda5607b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:34.814825 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-40fda719-ea41-4f4c-b74c-798dda5607b0 None None] MFA Rules not processed for user `b60f92ebcd344597adb384754595929f`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:34.816769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-40fda719-ea41-4f4c-b74c-798dda5607b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:34.914734 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1296/2613] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:34.932617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b5b63-c93a-47e2-b2e5-537b17f763f0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:34.935223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-456b5b63-c93a-47e2-b2e5-537b17f763f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:35.021477 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b5b63-c93a-47e2-b2e5-537b17f763f0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:35.026189 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41a38b-eb68-451a-89e3-20d6c9ce0129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:35.027242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41a38b-eb68-451a-89e3-20d6c9ce0129 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:35.027521 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41a38b-eb68-451a-89e3-20d6c9ce0129 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:35.027791 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41a38b-eb68-451a-89e3-20d6c9ce0129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:35.040460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41a38b-eb68-451a-89e3-20d6c9ce0129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:35.041508 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41a38b-eb68-451a-89e3-20d6c9ce0129 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:35.052402 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1318/2614] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:35.212233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fda719-ea41-4f4c-b74c-798dda5607b0 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:35.228061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fda719-ea41-4f4c-b74c-798dda5607b0 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:35.233992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7d0ca-12a9-4b9c-926f-21a4ed91d069 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:35.234999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7d0ca-12a9-4b9c-926f-21a4ed91d069 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:35.235465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7d0ca-12a9-4b9c-926f-21a4ed91d069 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:35.236008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7d0ca-12a9-4b9c-926f-21a4ed91d069 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:35.275719 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:35.325433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7d0ca-12a9-4b9c-926f-21a4ed91d069 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:35.330734 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7d0ca-12a9-4b9c-926f-21a4ed91d069 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:35.343939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1297/2615] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:38.168102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41a38b-eb68-451a-89e3-20d6c9ce0129 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:38.275305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41a38b-eb68-451a-89e3-20d6c9ce0129 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:38.281275 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5cbf6-9a43-4a64-a27d-a5e9cb55c80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:38.281928 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5cbf6-9a43-4a64-a27d-a5e9cb55c80c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:38.282203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5cbf6-9a43-4a64-a27d-a5e9cb55c80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:38.282469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5cbf6-9a43-4a64-a27d-a5e9cb55c80c admin admin] PATH_INFO: `/v3/users/0f44e04fc67b4ec4a0b013ab06e6bf0d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:38.283659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5cbf6-9a43-4a64-a27d-a5e9cb55c80c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f44e04fc67b4ec4a0b013ab06e6bf0d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:38.288006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5cbf6-9a43-4a64-a27d-a5e9cb55c80c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:38.350500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eee5cbf6-9a43-4a64-a27d-a5e9cb55c80c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f44e04fc67b4ec4a0b013ab06e6bf0d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:38.362476 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eee5cbf6-9a43-4a64-a27d-a5e9cb55c80c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f44e04fc67b4ec4a0b013ab06e6bf0d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:38.363403 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eee5cbf6-9a43-4a64-a27d-a5e9cb55c80c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f44e04fc67b4ec4a0b013ab06e6bf0d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:38.381383 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eee5cbf6-9a43-4a64-a27d-a5e9cb55c80c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f44e04fc67b4ec4a0b013ab06e6bf0d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:38.390087 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eee5cbf6-9a43-4a64-a27d-a5e9cb55c80c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f44e04fc67b4ec4a0b013ab06e6bf0d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:38.410788 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1319/2616] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:38 2022] DELETE /identity/v3/users/0f44e04fc67b4ec4a0b013ab06e6bf0d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:38.416864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996c035-7f17-4106-9c29-e3a204e6920e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:38.417518 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996c035-7f17-4106-9c29-e3a204e6920e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:38.417518 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996c035-7f17-4106-9c29-e3a204e6920e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:38.443950 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6996c035-7f17-4106-9c29-e3a204e6920e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:38.446375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6996c035-7f17-4106-9c29-e3a204e6920e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:38.531031 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1298/2617] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:38.545741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5cbf6-9a43-4a64-a27d-a5e9cb55c80c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:38.547299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eee5cbf6-9a43-4a64-a27d-a5e9cb55c80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:38.560582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:38.624878 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5cbf6-9a43-4a64-a27d-a5e9cb55c80c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:38.628614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fe290-6eba-4340-8397-f723153c6989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:38.629246 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fe290-6eba-4340-8397-f723153c6989 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:38.629520 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fe290-6eba-4340-8397-f723153c6989 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:38.629805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fe290-6eba-4340-8397-f723153c6989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:38.640705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fe290-6eba-4340-8397-f723153c6989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:38.642160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fe290-6eba-4340-8397-f723153c6989 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:38.653803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1320/2618] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:38.987743 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996c035-7f17-4106-9c29-e3a204e6920e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:38.990486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6996c035-7f17-4106-9c29-e3a204e6920e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:39.013116 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996c035-7f17-4106-9c29-e3a204e6920e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:39.021200 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:39.022309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:39.022737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:39.023198 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] PATH_INFO: `/v3/projects/394486dea38b4fbe8d070fc198aa4c6c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:39.026271 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] RBAC: Authorizing `identity:delete_project(project_id=394486dea38b4fbe8d070fc198aa4c6c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:39.029522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:39.096891 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '394486dea38b4fbe8d070fc198aa4c6c', 'request_id': 'req-9ed7b383-6110-4b07-baa7-3cb65c693ecf'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.105319 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '394486dea38b4fbe8d070fc198aa4c6c', 'request_id': 'req-9ed7b383-6110-4b07-baa7-3cb65c693ecf'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.111025 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '394486dea38b4fbe8d070fc198aa4c6c', 'request_id': 'req-9ed7b383-6110-4b07-baa7-3cb65c693ecf'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.124113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] The token cache is being invalidate because project 394486dea38b4fbe8d070fc198aa4c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:39.124545 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.125484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.127675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1299/2619] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:38 2022] DELETE /identity/v3/projects/394486dea38b4fbe8d070fc198aa4c6c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:39.133237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fe290-6eba-4340-8397-f723153c6989 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:39.146262 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fe290-6eba-4340-8397-f723153c6989 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:39.231259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2ee05-2981-4f49-a670-485ecab8cef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:39.232370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2ee05-2981-4f49-a670-485ecab8cef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:39.232742 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2ee05-2981-4f49-a670-485ecab8cef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:39.233300 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2ee05-2981-4f49-a670-485ecab8cef2 admin admin] PATH_INFO: `/v3/users/b60f92ebcd344597adb384754595929f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:39.235173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2ee05-2981-4f49-a670-485ecab8cef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b60f92ebcd344597adb384754595929f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:39.237399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2ee05-2981-4f49-a670-485ecab8cef2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:39.302896 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-faa2ee05-2981-4f49-a670-485ecab8cef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b60f92ebcd344597adb384754595929f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.313631 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-faa2ee05-2981-4f49-a670-485ecab8cef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b60f92ebcd344597adb384754595929f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.314328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-faa2ee05-2981-4f49-a670-485ecab8cef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b60f92ebcd344597adb384754595929f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.330222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-faa2ee05-2981-4f49-a670-485ecab8cef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b60f92ebcd344597adb384754595929f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.339794 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-faa2ee05-2981-4f49-a670-485ecab8cef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b60f92ebcd344597adb384754595929f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.349121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1321/2620] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:39 2022] DELETE /identity/v3/users/b60f92ebcd344597adb384754595929f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:39.714444 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:39.718605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:39.739023 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7b383-6110-4b07-baa7-3cb65c693ecf admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:39.798255 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:39.816418 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:39.817398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:39.817812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:39.818081 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c admin admin] PATH_INFO: `/v3/projects/1843694a25e74b58a409656e4a0750ab` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:39.821759 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1843694a25e74b58a409656e4a0750ab)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:39.825085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:39.880710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1843694a25e74b58a409656e4a0750ab', 'request_id': 'req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.890557 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1843694a25e74b58a409656e4a0750ab', 'request_id': 'req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.897061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1843694a25e74b58a409656e4a0750ab', 'request_id': 'req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.910023 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c admin admin] The token cache is being invalidate because project 1843694a25e74b58a409656e4a0750ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:39.910332 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.910973 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-33d89b46-8e3e-48cb-81fc-4aa75d98b51c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:39.912567 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1300/2621] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:39 2022] DELETE /identity/v3/projects/1843694a25e74b58a409656e4a0750ab => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:39.919867 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb73470-91e3-431f-9c7d-4a91f496374b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:39.920370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb73470-91e3-431f-9c7d-4a91f496374b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:39.920828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb73470-91e3-431f-9c7d-4a91f496374b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:39.944440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1eb73470-91e3-431f-9c7d-4a91f496374b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:39.945728 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1eb73470-91e3-431f-9c7d-4a91f496374b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:39.974099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:40.015077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1322/2622] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:40.025599 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ef3ef-1cb2-444c-b4a1-556a5f4ef4e4 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:40.026039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ef3ef-1cb2-444c-b4a1-556a5f4ef4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:40.026506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ef3ef-1cb2-444c-b4a1-556a5f4ef4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:40.054175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-127ef3ef-1cb2-444c-b4a1-556a5f4ef4e4 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:40.055845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-127ef3ef-1cb2-444c-b4a1-556a5f4ef4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:40.066179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1301/2623] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:40.071358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb73470-91e3-431f-9c7d-4a91f496374b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:40.082264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb73470-91e3-431f-9c7d-4a91f496374b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:40.088163 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1ed88-a51d-4816-bd7f-503b68e7b4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:40.088786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1ed88-a51d-4816-bd7f-503b68e7b4ff admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:40.089063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1ed88-a51d-4816-bd7f-503b68e7b4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:40.089323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1ed88-a51d-4816-bd7f-503b68e7b4ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:40.090014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1ed88-a51d-4816-bd7f-503b68e7b4ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:40.090349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1ed88-a51d-4816-bd7f-503b68e7b4ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:40.092181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1ed88-a51d-4816-bd7f-503b68e7b4ff admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:40.105548 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1323/2624] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:40.110656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ef3ef-1cb2-444c-b4a1-556a5f4ef4e4 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:40.122661 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ef3ef-1cb2-444c-b4a1-556a5f4ef4e4 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:40.127658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d3b29-a339-40a6-b5b2-ef9d571c6965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:40.128304 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d3b29-a339-40a6-b5b2-ef9d571c6965 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:40.128612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d3b29-a339-40a6-b5b2-ef9d571c6965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:40.128877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d3b29-a339-40a6-b5b2-ef9d571c6965 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:40.129511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d3b29-a339-40a6-b5b2-ef9d571c6965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:40.131444 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d3b29-a339-40a6-b5b2-ef9d571c6965 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:40.164536 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1302/2625] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:40 2022] POST /identity/v3/projects => generated 407 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:40.170960 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1ed88-a51d-4816-bd7f-503b68e7b4ff admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:40.182774 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1ed88-a51d-4816-bd7f-503b68e7b4ff admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:40.252713 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9121888-abf1-4df2-8787-ef97b15a35cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:40.253692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9121888-abf1-4df2-8787-ef97b15a35cf admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:40.253861 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9121888-abf1-4df2-8787-ef97b15a35cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:40.254150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9121888-abf1-4df2-8787-ef97b15a35cf admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:40.254906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9121888-abf1-4df2-8787-ef97b15a35cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:40.257159 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9121888-abf1-4df2-8787-ef97b15a35cf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:40.283717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1324/2626] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:40 2022] POST /identity/v3/users => generated 304 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:40.289861 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d3b29-a339-40a6-b5b2-ef9d571c6965 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:40.302192 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d3b29-a339-40a6-b5b2-ef9d571c6965 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:40.311868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9fe3a-e415-416a-8248-f0eacd7a5344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:40.313035 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9fe3a-e415-416a-8248-f0eacd7a5344 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:40.313523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9fe3a-e415-416a-8248-f0eacd7a5344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:40.314012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9fe3a-e415-416a-8248-f0eacd7a5344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:40.315191 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9fe3a-e415-416a-8248-f0eacd7a5344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:40.318465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9fe3a-e415-416a-8248-f0eacd7a5344 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:40.332820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1303/2627] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:40.339317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9121888-abf1-4df2-8787-ef97b15a35cf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:40.356645 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9121888-abf1-4df2-8787-ef97b15a35cf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:40.364147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f1398-1a3f-4522-840e-194f56e8e847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:40.364762 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f1398-1a3f-4522-840e-194f56e8e847 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:40.365045 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f1398-1a3f-4522-840e-194f56e8e847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:40.365308 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f1398-1a3f-4522-840e-194f56e8e847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:40.366125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f1398-1a3f-4522-840e-194f56e8e847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:40.368121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f1398-1a3f-4522-840e-194f56e8e847 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:40.383984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1325/2628] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:40.393894 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9fe3a-e415-416a-8248-f0eacd7a5344 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:40.405862 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9fe3a-e415-416a-8248-f0eacd7a5344 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:40.411473 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c759204-88c3-49ba-973a-41cd497cec3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:40.412197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c759204-88c3-49ba-973a-41cd497cec3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:40.412469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c759204-88c3-49ba-973a-41cd497cec3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:40.412798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c759204-88c3-49ba-973a-41cd497cec3e admin admin] PATH_INFO: `/v3/projects/78110234c61d4064bddb827f6ff4220d/users/eaec298d090444e9b737c1036ce8f31b/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:40.429913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c759204-88c3-49ba-973a-41cd497cec3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=78110234c61d4064bddb827f6ff4220d, user_id=eaec298d090444e9b737c1036ce8f31b, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:40.432959 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c759204-88c3-49ba-973a-41cd497cec3e admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:40.452065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1304/2629] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:40 2022] PUT /identity/v3/projects/78110234c61d4064bddb827f6ff4220d/users/eaec298d090444e9b737c1036ce8f31b/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:40.531506 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01447800-032c-4094-92c5-2a4c4078d13b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:40.531996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01447800-032c-4094-92c5-2a4c4078d13b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:40.532473 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01447800-032c-4094-92c5-2a4c4078d13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:40.584105 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-01447800-032c-4094-92c5-2a4c4078d13b None None] MFA Rules not processed for user `eaec298d090444e9b737c1036ce8f31b`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:40.585546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-01447800-032c-4094-92c5-2a4c4078d13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:40.715729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1326/2630] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:40 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:40.718013 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c759204-88c3-49ba-973a-41cd497cec3e admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:40.826626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c759204-88c3-49ba-973a-41cd497cec3e admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:40.834094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff6074-5a69-4190-a97c-ac6bdd6108b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:40.835070 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff6074-5a69-4190-a97c-ac6bdd6108b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:40.835586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff6074-5a69-4190-a97c-ac6bdd6108b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:40.836110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff6074-5a69-4190-a97c-ac6bdd6108b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:40.837596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff6074-5a69-4190-a97c-ac6bdd6108b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:40.840100 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff6074-5a69-4190-a97c-ac6bdd6108b2 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:40.873395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:40.894167 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1305/2631] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:40 2022] POST /identity/v3/projects => generated 407 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:40.900237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01447800-032c-4094-92c5-2a4c4078d13b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:40.913925 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01447800-032c-4094-92c5-2a4c4078d13b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:40.950366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:41.028916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539f6bf-0378-4167-a4f7-8c058fd4a62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:41.030697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539f6bf-0378-4167-a4f7-8c058fd4a62b admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:41.031182 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539f6bf-0378-4167-a4f7-8c058fd4a62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:41.031549 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539f6bf-0378-4167-a4f7-8c058fd4a62b admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:41.032481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539f6bf-0378-4167-a4f7-8c058fd4a62b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:41.036075 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539f6bf-0378-4167-a4f7-8c058fd4a62b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:41.084348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1327/2632] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:40 2022] POST /identity/v3/users => generated 310 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:41.088094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff6074-5a69-4190-a97c-ac6bdd6108b2 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:41.102186 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff6074-5a69-4190-a97c-ac6bdd6108b2 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:41.107561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2427324-10fd-4f54-91dd-4219c35b6715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:41.108378 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2427324-10fd-4f54-91dd-4219c35b6715 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:41.108584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2427324-10fd-4f54-91dd-4219c35b6715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:41.108957 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2427324-10fd-4f54-91dd-4219c35b6715 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:41.109510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2427324-10fd-4f54-91dd-4219c35b6715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:41.111135 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2427324-10fd-4f54-91dd-4219c35b6715 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:41.125576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1306/2633] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:41.129705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539f6bf-0378-4167-a4f7-8c058fd4a62b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:41.141315 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539f6bf-0378-4167-a4f7-8c058fd4a62b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:41.146964 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d889a-4e75-410c-bc38-bb266f03664a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:41.148491 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d889a-4e75-410c-bc38-bb266f03664a admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:41.148491 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d889a-4e75-410c-bc38-bb266f03664a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:41.148491 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d889a-4e75-410c-bc38-bb266f03664a admin admin] PATH_INFO: `/v3/projects/432b028d7e3b4440bd2a0564d6108e69/users/5dd8cfde2663410caca67c2d4d2103fc/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:41.176165 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d889a-4e75-410c-bc38-bb266f03664a admin admin] RBAC: Authorizing `identity:create_grant(project_id=432b028d7e3b4440bd2a0564d6108e69, user_id=5dd8cfde2663410caca67c2d4d2103fc, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:41.183492 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d889a-4e75-410c-bc38-bb266f03664a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:41.207460 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1328/2634] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:41 2022] PUT /identity/v3/projects/432b028d7e3b4440bd2a0564d6108e69/users/5dd8cfde2663410caca67c2d4d2103fc/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:41.215546 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2427324-10fd-4f54-91dd-4219c35b6715 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:41.229315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2427324-10fd-4f54-91dd-4219c35b6715 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:41.346462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c098d18-cf85-4ef7-b185-9c20e7f16da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:41.347528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c098d18-cf85-4ef7-b185-9c20e7f16da9 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:41.348012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c098d18-cf85-4ef7-b185-9c20e7f16da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:41.348270 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c098d18-cf85-4ef7-b185-9c20e7f16da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:41.348977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c098d18-cf85-4ef7-b185-9c20e7f16da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:41.351224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c098d18-cf85-4ef7-b185-9c20e7f16da9 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:41.364467 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1307/2635] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:41.371271 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d889a-4e75-410c-bc38-bb266f03664a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:41.390944 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d889a-4e75-410c-bc38-bb266f03664a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:41.398937 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25981cc-982a-442b-bc9c-9e939a6f25dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:41.400351 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25981cc-982a-442b-bc9c-9e939a6f25dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:41.400849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25981cc-982a-442b-bc9c-9e939a6f25dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:41.401281 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25981cc-982a-442b-bc9c-9e939a6f25dc admin admin] PATH_INFO: `/v3/projects/432b028d7e3b4440bd2a0564d6108e69/users/5dd8cfde2663410caca67c2d4d2103fc/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:41.407091 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25981cc-982a-442b-bc9c-9e939a6f25dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=432b028d7e3b4440bd2a0564d6108e69, user_id=5dd8cfde2663410caca67c2d4d2103fc, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:41.411788 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25981cc-982a-442b-bc9c-9e939a6f25dc admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:41.436257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1329/2636] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:41 2022] PUT /identity/v3/projects/432b028d7e3b4440bd2a0564d6108e69/users/5dd8cfde2663410caca67c2d4d2103fc/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:41.451210 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1918a7-f6fc-4b46-b579-037f8b73f730 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:41.451687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1918a7-f6fc-4b46-b579-037f8b73f730 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:41.452275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1918a7-f6fc-4b46-b579-037f8b73f730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:41.510888 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-ef1918a7-f6fc-4b46-b579-037f8b73f730 None None] MFA Rules not processed for user `5dd8cfde2663410caca67c2d4d2103fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:41.512263 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ef1918a7-f6fc-4b46-b579-037f8b73f730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:41.644903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1308/2637] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:41 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:41.660414 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25981cc-982a-442b-bc9c-9e939a6f25dc admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:41.663046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f25981cc-982a-442b-bc9c-9e939a6f25dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:41.799251 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25981cc-982a-442b-bc9c-9e939a6f25dc admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:41.815133 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92652421-c55c-4542-90ec-dbab32c848d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:41.816011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92652421-c55c-4542-90ec-dbab32c848d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:41.816433 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92652421-c55c-4542-90ec-dbab32c848d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:41.817213 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92652421-c55c-4542-90ec-dbab32c848d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:41.835536 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92652421-c55c-4542-90ec-dbab32c848d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:41.839668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92652421-c55c-4542-90ec-dbab32c848d4 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:41.859764 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1330/2638] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:41 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:42.130055 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1918a7-f6fc-4b46-b579-037f8b73f730 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:42.145739 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1918a7-f6fc-4b46-b579-037f8b73f730 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:42.159773 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d3523-6935-4601-951b-c89215898bef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:42.160661 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d3523-6935-4601-951b-c89215898bef service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:42.161128 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d3523-6935-4601-951b-c89215898bef service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:42.161593 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d3523-6935-4601-951b-c89215898bef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:42.238469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d3523-6935-4601-951b-c89215898bef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:42.242169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d3523-6935-4601-951b-c89215898bef service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:42.266249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1309/2639] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:42 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:42.347176 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92652421-c55c-4542-90ec-dbab32c848d4 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:42.349098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-92652421-c55c-4542-90ec-dbab32c848d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:42.450505 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92652421-c55c-4542-90ec-dbab32c848d4 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:42.457505 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac19c23-c822-4cce-83f1-986d2826a106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:42.458283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac19c23-c822-4cce-83f1-986d2826a106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:42.458718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac19c23-c822-4cce-83f1-986d2826a106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:42.459076 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac19c23-c822-4cce-83f1-986d2826a106 admin admin] PATH_INFO: `/v3/users/542afb2e8b074fbd873969988fc3d849` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:42.460633 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac19c23-c822-4cce-83f1-986d2826a106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=542afb2e8b074fbd873969988fc3d849)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:42.462883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac19c23-c822-4cce-83f1-986d2826a106 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:42.494877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:42.538482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bac19c23-c822-4cce-83f1-986d2826a106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '542afb2e8b074fbd873969988fc3d849'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:42.552139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bac19c23-c822-4cce-83f1-986d2826a106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '542afb2e8b074fbd873969988fc3d849'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:42.553120 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bac19c23-c822-4cce-83f1-986d2826a106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '542afb2e8b074fbd873969988fc3d849'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:42.574240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bac19c23-c822-4cce-83f1-986d2826a106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '542afb2e8b074fbd873969988fc3d849'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:42.582671 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bac19c23-c822-4cce-83f1-986d2826a106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '542afb2e8b074fbd873969988fc3d849'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:42.590799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1331/2640] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:42 2022] DELETE /identity/v3/users/542afb2e8b074fbd873969988fc3d849 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:42.865563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d3523-6935-4601-951b-c89215898bef service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:42.881465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d3523-6935-4601-951b-c89215898bef service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:42.886126 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:42.978596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:42.979564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:42.980083 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:42.980521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] PATH_INFO: `/v3/projects/ffbf1587d4ca4f379ef3c89a47cc42cd` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:42.982249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffbf1587d4ca4f379ef3c89a47cc42cd)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:42.988347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:43.064577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffbf1587d4ca4f379ef3c89a47cc42cd', 'request_id': 'req-d33baed5-bca2-4aa5-9539-6f52e3d9f254'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.075760 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffbf1587d4ca4f379ef3c89a47cc42cd', 'request_id': 'req-d33baed5-bca2-4aa5-9539-6f52e3d9f254'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.084857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffbf1587d4ca4f379ef3c89a47cc42cd', 'request_id': 'req-d33baed5-bca2-4aa5-9539-6f52e3d9f254'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.091102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac19c23-c822-4cce-83f1-986d2826a106 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:43.093318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bac19c23-c822-4cce-83f1-986d2826a106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:43.105503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] The token cache is being invalidate because project ffbf1587d4ca4f379ef3c89a47cc42cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:43.106126 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.106841 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.108884 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1310/2641] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:42 2022] DELETE /identity/v3/projects/ffbf1587d4ca4f379ef3c89a47cc42cd => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:43.111554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac19c23-c822-4cce-83f1-986d2826a106 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:43.113460 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:43.115416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:43.193514 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33baed5-bca2-4aa5-9539-6f52e3d9f254 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:43.197854 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9770bc-78fc-4f2a-a106-63e73a9e551b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:43.198654 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9770bc-78fc-4f2a-a106-63e73a9e551b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:43.199033 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9770bc-78fc-4f2a-a106-63e73a9e551b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:43.199482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9770bc-78fc-4f2a-a106-63e73a9e551b admin admin] PATH_INFO: `/v3/users/11f63b2804304a5f8274dffd5e8a602d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:43.199764 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c9bf0-0bf1-4715-a08c-adb4c263cd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:43.200561 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c9bf0-0bf1-4715-a08c-adb4c263cd5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:43.200899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c9bf0-0bf1-4715-a08c-adb4c263cd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:43.201086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9770bc-78fc-4f2a-a106-63e73a9e551b admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f63b2804304a5f8274dffd5e8a602d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:43.201250 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c9bf0-0bf1-4715-a08c-adb4c263cd5a admin admin] PATH_INFO: `/v3/users/51d75351afea431da240fa50df845a9e` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:43.202611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c9bf0-0bf1-4715-a08c-adb4c263cd5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=51d75351afea431da240fa50df845a9e)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:43.203347 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9770bc-78fc-4f2a-a106-63e73a9e551b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:43.204924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c9bf0-0bf1-4715-a08c-adb4c263cd5a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:43.265267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:43.273405 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7b7c9bf0-0bf1-4715-a08c-adb4c263cd5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51d75351afea431da240fa50df845a9e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.274104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-af9770bc-78fc-4f2a-a106-63e73a9e551b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f63b2804304a5f8274dffd5e8a602d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.285431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-af9770bc-78fc-4f2a-a106-63e73a9e551b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f63b2804304a5f8274dffd5e8a602d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.286147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-af9770bc-78fc-4f2a-a106-63e73a9e551b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f63b2804304a5f8274dffd5e8a602d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.297004 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7b7c9bf0-0bf1-4715-a08c-adb4c263cd5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51d75351afea431da240fa50df845a9e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.304325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-af9770bc-78fc-4f2a-a106-63e73a9e551b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f63b2804304a5f8274dffd5e8a602d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.309685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7b7c9bf0-0bf1-4715-a08c-adb4c263cd5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51d75351afea431da240fa50df845a9e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.313847 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-af9770bc-78fc-4f2a-a106-63e73a9e551b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f63b2804304a5f8274dffd5e8a602d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.323415 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1332/2642] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:43 2022] DELETE /identity/v3/users/11f63b2804304a5f8274dffd5e8a602d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:43.328605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7b7c9bf0-0bf1-4715-a08c-adb4c263cd5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51d75351afea431da240fa50df845a9e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.329704 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7b7c9bf0-0bf1-4715-a08c-adb4c263cd5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51d75351afea431da240fa50df845a9e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.330106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f8695-a9d4-415d-b3c5-7fc7a837900a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:43.330457 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f8695-a9d4-415d-b3c5-7fc7a837900a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:43.330726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f8695-a9d4-415d-b3c5-7fc7a837900a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:43.337310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1311/2643] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:43 2022] DELETE /identity/v3/users/51d75351afea431da240fa50df845a9e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:43.358557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e70f8695-a9d4-415d-b3c5-7fc7a837900a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:43.360923 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e70f8695-a9d4-415d-b3c5-7fc7a837900a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:43.379496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:43.465701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1333/2644] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:43.483667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c9bf0-0bf1-4715-a08c-adb4c263cd5a admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:43.486135 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7b7c9bf0-0bf1-4715-a08c-adb4c263cd5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:43.580285 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f8695-a9d4-415d-b3c5-7fc7a837900a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:43.581975 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e70f8695-a9d4-415d-b3c5-7fc7a837900a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:43.585121 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c9bf0-0bf1-4715-a08c-adb4c263cd5a admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:43.591376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb180759-22dd-49d4-8848-2c5465932977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:43.592035 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb180759-22dd-49d4-8848-2c5465932977 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:43.592475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb180759-22dd-49d4-8848-2c5465932977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:43.592948 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb180759-22dd-49d4-8848-2c5465932977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:43.597544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f8695-a9d4-415d-b3c5-7fc7a837900a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:43.603701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55e385-85d7-44bb-a848-59a870277cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:43.604373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55e385-85d7-44bb-a848-59a870277cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:43.604651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55e385-85d7-44bb-a848-59a870277cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:43.604921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55e385-85d7-44bb-a848-59a870277cc1 admin admin] PATH_INFO: `/v3/projects/8a1ba30664044d5b8ea1782c12a529b2` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:43.606175 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55e385-85d7-44bb-a848-59a870277cc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a1ba30664044d5b8ea1782c12a529b2)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:43.607674 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb180759-22dd-49d4-8848-2c5465932977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:43.608136 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55e385-85d7-44bb-a848-59a870277cc1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:43.609998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb180759-22dd-49d4-8848-2c5465932977 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:43.621058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1312/2645] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:43.672051 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5c55e385-85d7-44bb-a848-59a870277cc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a1ba30664044d5b8ea1782c12a529b2', 'request_id': 'req-5c55e385-85d7-44bb-a848-59a870277cc1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.679665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5c55e385-85d7-44bb-a848-59a870277cc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a1ba30664044d5b8ea1782c12a529b2', 'request_id': 'req-5c55e385-85d7-44bb-a848-59a870277cc1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.691580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5c55e385-85d7-44bb-a848-59a870277cc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a1ba30664044d5b8ea1782c12a529b2', 'request_id': 'req-5c55e385-85d7-44bb-a848-59a870277cc1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.707970 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5c55e385-85d7-44bb-a848-59a870277cc1 admin admin] The token cache is being invalidate because project 8a1ba30664044d5b8ea1782c12a529b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:43.708265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5c55e385-85d7-44bb-a848-59a870277cc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.709126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5c55e385-85d7-44bb-a848-59a870277cc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:43.714405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1334/2646] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:43 2022] DELETE /identity/v3/projects/8a1ba30664044d5b8ea1782c12a529b2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:43.716109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a097e-90ff-4254-b700-029be18321af None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:43.716391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a097e-90ff-4254-b700-029be18321af None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:43.716774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a097e-90ff-4254-b700-029be18321af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:43.746644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-743a097e-90ff-4254-b700-029be18321af None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:43.749947 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-743a097e-90ff-4254-b700-029be18321af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:43.880073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1313/2647] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:43.893357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e893b-499e-4444-a3ff-4734b138ce25 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:43.893842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e893b-499e-4444-a3ff-4734b138ce25 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:43.894316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e893b-499e-4444-a3ff-4734b138ce25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:43.930261 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-586e893b-499e-4444-a3ff-4734b138ce25 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:43.932446 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-586e893b-499e-4444-a3ff-4734b138ce25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:43.957096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1335/2648] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:43.963274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a097e-90ff-4254-b700-029be18321af None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:43.973907 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a097e-90ff-4254-b700-029be18321af None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:43.980027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bdc1d-96ec-4769-ae1c-dad787a73364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:43.981178 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bdc1d-96ec-4769-ae1c-dad787a73364 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:43.981618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bdc1d-96ec-4769-ae1c-dad787a73364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:43.981618 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bdc1d-96ec-4769-ae1c-dad787a73364 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:43.982500 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bdc1d-96ec-4769-ae1c-dad787a73364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:43.983072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bdc1d-96ec-4769-ae1c-dad787a73364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:43.984446 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bdc1d-96ec-4769-ae1c-dad787a73364 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:44.002356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1314/2649] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:44.009940 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e893b-499e-4444-a3ff-4734b138ce25 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:44.039009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e893b-499e-4444-a3ff-4734b138ce25 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:44.046858 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6f0d8-85e3-488a-bdb4-8e292ff990b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:44.048310 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6f0d8-85e3-488a-bdb4-8e292ff990b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:44.048905 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6f0d8-85e3-488a-bdb4-8e292ff990b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:44.049325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6f0d8-85e3-488a-bdb4-8e292ff990b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:44.050551 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6f0d8-85e3-488a-bdb4-8e292ff990b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:44.054358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6f0d8-85e3-488a-bdb4-8e292ff990b5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:44.073372 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bdc1d-96ec-4769-ae1c-dad787a73364 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:44.077375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-350bdc1d-96ec-4769-ae1c-dad787a73364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:44.104934 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bdc1d-96ec-4769-ae1c-dad787a73364 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:44.116038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1336/2650] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:44 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:44.119262 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:44.120301 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:44.120778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:44.121041 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6f0d8-85e3-488a-bdb4-8e292ff990b5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:44.121256 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] PATH_INFO: `/v3/projects/2166eb7e8d1344eda9d963b32a8b4f0d` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:44.123177 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2166eb7e8d1344eda9d963b32a8b4f0d)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:44.126775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:44.191197 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:44.203061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2166eb7e8d1344eda9d963b32a8b4f0d', 'request_id': 'req-337a5864-3102-407a-81fc-0e69dab5e189'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:44.215499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2166eb7e8d1344eda9d963b32a8b4f0d', 'request_id': 'req-337a5864-3102-407a-81fc-0e69dab5e189'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:44.216523 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6f0d8-85e3-488a-bdb4-8e292ff990b5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:44.223297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2166eb7e8d1344eda9d963b32a8b4f0d', 'request_id': 'req-337a5864-3102-407a-81fc-0e69dab5e189'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:44.238063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] The token cache is being invalidate because project 2166eb7e8d1344eda9d963b32a8b4f0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:44.238376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:44.239030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:44.240216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:44.240624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1315/2651] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:44 2022] DELETE /identity/v3/projects/2166eb7e8d1344eda9d963b32a8b4f0d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:44.246058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:44.247441 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:44.303938 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ecf65-030f-4efd-be75-0be7dcb50f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:44.305276 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ecf65-030f-4efd-be75-0be7dcb50f28 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:44.305735 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ecf65-030f-4efd-be75-0be7dcb50f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:44.306098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ecf65-030f-4efd-be75-0be7dcb50f28 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:44.307080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ecf65-030f-4efd-be75-0be7dcb50f28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:44.309386 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ecf65-030f-4efd-be75-0be7dcb50f28 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:44.334013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1337/2652] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:44 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:44.339066 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a5864-3102-407a-81fc-0e69dab5e189 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:44.339752 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ecf65-030f-4efd-be75-0be7dcb50f28 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:44.342510 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f28ecf65-030f-4efd-be75-0be7dcb50f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:44.346519 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69bb79-ab8d-400c-b747-5ac696ca6847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:44.347515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69bb79-ab8d-400c-b747-5ac696ca6847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:44.347956 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69bb79-ab8d-400c-b747-5ac696ca6847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:44.348388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69bb79-ab8d-400c-b747-5ac696ca6847 admin admin] PATH_INFO: `/v3/users/9a4f0fc2b26441bcaa9ccb4fdc03685e` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:44.350192 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69bb79-ab8d-400c-b747-5ac696ca6847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a4f0fc2b26441bcaa9ccb4fdc03685e)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:44.353650 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69bb79-ab8d-400c-b747-5ac696ca6847 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:44.361370 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ecf65-030f-4efd-be75-0be7dcb50f28 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:44.368056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b346d-bfbe-4cba-9ea6-3428830b4aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:44.368927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b346d-bfbe-4cba-9ea6-3428830b4aab admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:44.369214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b346d-bfbe-4cba-9ea6-3428830b4aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:44.369486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b346d-bfbe-4cba-9ea6-3428830b4aab admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:44.370196 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b346d-bfbe-4cba-9ea6-3428830b4aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:44.371833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b346d-bfbe-4cba-9ea6-3428830b4aab admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:44.401939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1338/2653] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:44.405856 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b346d-bfbe-4cba-9ea6-3428830b4aab admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:44.424089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b346d-bfbe-4cba-9ea6-3428830b4aab admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:44.429952 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc684d3c-f889-4c86-8f22-c6feafbbe17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:44.430584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc684d3c-f889-4c86-8f22-c6feafbbe17d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:44.430862 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc684d3c-f889-4c86-8f22-c6feafbbe17d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:44.431133 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc684d3c-f889-4c86-8f22-c6feafbbe17d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:44.432244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc684d3c-f889-4c86-8f22-c6feafbbe17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:44.433746 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc684d3c-f889-4c86-8f22-c6feafbbe17d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:44.439516 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4e69bb79-ab8d-400c-b747-5ac696ca6847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a4f0fc2b26441bcaa9ccb4fdc03685e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:44.448449 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1339/2654] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:44.453978 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc684d3c-f889-4c86-8f22-c6feafbbe17d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:44.461462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4e69bb79-ab8d-400c-b747-5ac696ca6847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a4f0fc2b26441bcaa9ccb4fdc03685e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:44.471291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc684d3c-f889-4c86-8f22-c6feafbbe17d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:44.472896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4e69bb79-ab8d-400c-b747-5ac696ca6847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a4f0fc2b26441bcaa9ccb4fdc03685e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:44.481406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c155a1e-a7ba-4b11-9076-9a68ca34cbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:44.482201 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c155a1e-a7ba-4b11-9076-9a68ca34cbdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:44.482516 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c155a1e-a7ba-4b11-9076-9a68ca34cbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:44.482919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c155a1e-a7ba-4b11-9076-9a68ca34cbdd admin admin] PATH_INFO: `/v3/projects/dad91d3c81754c309b0f685fd388e599/users/eff99b0038464462a133b175245494cb/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:44.486475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4e69bb79-ab8d-400c-b747-5ac696ca6847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a4f0fc2b26441bcaa9ccb4fdc03685e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:44.487680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4e69bb79-ab8d-400c-b747-5ac696ca6847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a4f0fc2b26441bcaa9ccb4fdc03685e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:44.498814 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1316/2655] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:44 2022] DELETE /identity/v3/users/9a4f0fc2b26441bcaa9ccb4fdc03685e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:44.510820 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c155a1e-a7ba-4b11-9076-9a68ca34cbdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad91d3c81754c309b0f685fd388e599, user_id=eff99b0038464462a133b175245494cb, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:44.515939 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c155a1e-a7ba-4b11-9076-9a68ca34cbdd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:44.534802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1340/2656] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:44 2022] PUT /identity/v3/projects/dad91d3c81754c309b0f685fd388e599/users/eff99b0038464462a133b175245494cb/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:44.546600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060c188-d840-4cfd-81bc-08c5a5b481c5 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:44.546883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060c188-d840-4cfd-81bc-08c5a5b481c5 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:44.547148 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060c188-d840-4cfd-81bc-08c5a5b481c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:44.593330 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6060c188-d840-4cfd-81bc-08c5a5b481c5 None None] MFA Rules not processed for user `eff99b0038464462a133b175245494cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:44.595164 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6060c188-d840-4cfd-81bc-08c5a5b481c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:44.663226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:44.706112 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1317/2657] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:44.713359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c155a1e-a7ba-4b11-9076-9a68ca34cbdd admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:44.716545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9c155a1e-a7ba-4b11-9076-9a68ca34cbdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:44.827951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c155a1e-a7ba-4b11-9076-9a68ca34cbdd admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:44.834094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10021dbb-d221-4e32-8151-d4054b88ce93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:44.834766 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10021dbb-d221-4e32-8151-d4054b88ce93 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:44.835035 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10021dbb-d221-4e32-8151-d4054b88ce93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:44.835296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10021dbb-d221-4e32-8151-d4054b88ce93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:44.836262 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10021dbb-d221-4e32-8151-d4054b88ce93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:44.840073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10021dbb-d221-4e32-8151-d4054b88ce93 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:44.876397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060c188-d840-4cfd-81bc-08c5a5b481c5 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:44.878180 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6060c188-d840-4cfd-81bc-08c5a5b481c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:44.881132 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1341/2658] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:44 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:44.887620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10021dbb-d221-4e32-8151-d4054b88ce93 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:44.902781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060c188-d840-4cfd-81bc-08c5a5b481c5 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:44.980855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:44.997723 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10021dbb-d221-4e32-8151-d4054b88ce93 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:45.001513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c006c78-5f3b-442f-91c8-1080b7a7e803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:45.002483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c006c78-5f3b-442f-91c8-1080b7a7e803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.002931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c006c78-5f3b-442f-91c8-1080b7a7e803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.003381 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c006c78-5f3b-442f-91c8-1080b7a7e803 admin admin] PATH_INFO: `/v3/projects/2188bfa324394fb08f59a8e334d03302` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.003817 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec51b6-e1c6-4a58-a482-d878ca857cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:45.004754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec51b6-e1c6-4a58-a482-d878ca857cee admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.005086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec51b6-e1c6-4a58-a482-d878ca857cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.005416 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec51b6-e1c6-4a58-a482-d878ca857cee admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.005646 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c006c78-5f3b-442f-91c8-1080b7a7e803 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2188bfa324394fb08f59a8e334d03302)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:45.006230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec51b6-e1c6-4a58-a482-d878ca857cee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:45.007615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c006c78-5f3b-442f-91c8-1080b7a7e803 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:45.009504 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec51b6-e1c6-4a58-a482-d878ca857cee admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:45.052952 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1342/2659] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:44 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:45.058249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec51b6-e1c6-4a58-a482-d878ca857cee admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:45.072253 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec51b6-e1c6-4a58-a482-d878ca857cee admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:45.078779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc123c-2bec-4ade-8087-35dc638a67f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:45.079807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc123c-2bec-4ade-8087-35dc638a67f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.080305 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc123c-2bec-4ade-8087-35dc638a67f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.081014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc123c-2bec-4ade-8087-35dc638a67f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.081694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc123c-2bec-4ade-8087-35dc638a67f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:45.084619 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc123c-2bec-4ade-8087-35dc638a67f9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:45.105811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1343/2660] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:45.110028 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc123c-2bec-4ade-8087-35dc638a67f9 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:45.116398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6c006c78-5f3b-442f-91c8-1080b7a7e803 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2188bfa324394fb08f59a8e334d03302', 'request_id': 'req-6c006c78-5f3b-442f-91c8-1080b7a7e803'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:45.122827 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc123c-2bec-4ade-8087-35dc638a67f9 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:45.129639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6c006c78-5f3b-442f-91c8-1080b7a7e803 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2188bfa324394fb08f59a8e334d03302', 'request_id': 'req-6c006c78-5f3b-442f-91c8-1080b7a7e803'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:45.141482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6c006c78-5f3b-442f-91c8-1080b7a7e803 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2188bfa324394fb08f59a8e334d03302', 'request_id': 'req-6c006c78-5f3b-442f-91c8-1080b7a7e803'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:45.154589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6c006c78-5f3b-442f-91c8-1080b7a7e803 admin admin] The token cache is being invalidate because project 2188bfa324394fb08f59a8e334d03302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:45.155053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6c006c78-5f3b-442f-91c8-1080b7a7e803 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:45.155813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6c006c78-5f3b-442f-91c8-1080b7a7e803 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:45.158133 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1318/2661] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:44 2022] DELETE /identity/v3/projects/2188bfa324394fb08f59a8e334d03302 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:45.166776 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55bef5-5791-4835-a17d-5d443724e196 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.167148 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55bef5-5791-4835-a17d-5d443724e196 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.167547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55bef5-5791-4835-a17d-5d443724e196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.194774 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-ea55bef5-5791-4835-a17d-5d443724e196 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:45.196757 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ea55bef5-5791-4835-a17d-5d443724e196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:45.210021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9587a72-8008-49f1-bfae-5fd89ef58825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:45.212336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9587a72-8008-49f1-bfae-5fd89ef58825 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.212721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9587a72-8008-49f1-bfae-5fd89ef58825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.212891 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9587a72-8008-49f1-bfae-5fd89ef58825 admin admin] PATH_INFO: `/v3/projects/33f8517a4cc04bc085185a35bb67eab2/users/24f3ccc2a2d642e89fe346316988a238/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.237146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9587a72-8008-49f1-bfae-5fd89ef58825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33f8517a4cc04bc085185a35bb67eab2, user_id=24f3ccc2a2d642e89fe346316988a238, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:45.239851 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9587a72-8008-49f1-bfae-5fd89ef58825 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:45.260413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1344/2662] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:45 2022] PUT /identity/v3/projects/33f8517a4cc04bc085185a35bb67eab2/users/24f3ccc2a2d642e89fe346316988a238/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:45.266104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9587a72-8008-49f1-bfae-5fd89ef58825 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:45.268616 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a9587a72-8008-49f1-bfae-5fd89ef58825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:45.271300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1319/2663] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:45.282874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed2cc5-6f4a-414f-8e10-76b463e8edda None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.283283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed2cc5-6f4a-414f-8e10-76b463e8edda None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.283647 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed2cc5-6f4a-414f-8e10-76b463e8edda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.308767 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e0ed2cc5-6f4a-414f-8e10-76b463e8edda None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:45.310146 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e0ed2cc5-6f4a-414f-8e10-76b463e8edda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:45.317775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:45.377399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9587a72-8008-49f1-bfae-5fd89ef58825 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:45.383027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c2129-da68-4b0a-9225-c1f06450594b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:45.384245 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c2129-da68-4b0a-9225-c1f06450594b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.384620 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c2129-da68-4b0a-9225-c1f06450594b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.385001 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c2129-da68-4b0a-9225-c1f06450594b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.385814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c2129-da68-4b0a-9225-c1f06450594b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:45.387659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c2129-da68-4b0a-9225-c1f06450594b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:45.398918 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1345/2664] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:45.405866 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c2129-da68-4b0a-9225-c1f06450594b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:45.406222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1320/2665] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:45.411402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed2cc5-6f4a-414f-8e10-76b463e8edda None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:45.423945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed2cc5-6f4a-414f-8e10-76b463e8edda None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:45.424261 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c2129-da68-4b0a-9225-c1f06450594b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:45.431982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6aded4-7ca6-4590-bfd2-0e30fa74a1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:45.432752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e42d69-110f-4c30-b4a4-c3536710acb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:45.432969 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aded4-7ca6-4590-bfd2-0e30fa74a1a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.433203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aded4-7ca6-4590-bfd2-0e30fa74a1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.433550 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aded4-7ca6-4590-bfd2-0e30fa74a1a7 admin admin] PATH_INFO: `/v3/projects/33f8517a4cc04bc085185a35bb67eab2/users/24f3ccc2a2d642e89fe346316988a238/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.433826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e42d69-110f-4c30-b4a4-c3536710acb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.434304 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e42d69-110f-4c30-b4a4-c3536710acb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.434789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e42d69-110f-4c30-b4a4-c3536710acb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.436034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e42d69-110f-4c30-b4a4-c3536710acb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:45.436417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e42d69-110f-4c30-b4a4-c3536710acb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:45.436740 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6aded4-7ca6-4590-bfd2-0e30fa74a1a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33f8517a4cc04bc085185a35bb67eab2, user_id=24f3ccc2a2d642e89fe346316988a238, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:45.439171 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e42d69-110f-4c30-b4a4-c3536710acb2 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:45.442608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6aded4-7ca6-4590-bfd2-0e30fa74a1a7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:45.459458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1346/2666] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:45 2022] PUT /identity/v3/projects/33f8517a4cc04bc085185a35bb67eab2/users/24f3ccc2a2d642e89fe346316988a238/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:45.460146 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1321/2667] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:45.466074 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6aded4-7ca6-4590-bfd2-0e30fa74a1a7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:45.470995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89ceff-f7cb-4bbe-a7bd-45e48d816472 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.471279 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89ceff-f7cb-4bbe-a7bd-45e48d816472 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.471571 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89ceff-f7cb-4bbe-a7bd-45e48d816472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.510214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-7f89ceff-f7cb-4bbe-a7bd-45e48d816472 None None] MFA Rules not processed for user `24f3ccc2a2d642e89fe346316988a238`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:45.511546 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7f89ceff-f7cb-4bbe-a7bd-45e48d816472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:45.535143 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:45.568532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6aded4-7ca6-4590-bfd2-0e30fa74a1a7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:45.576440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74906ee8-b80d-4247-b21c-112c023762d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:45.577437 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74906ee8-b80d-4247-b21c-112c023762d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.577880 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74906ee8-b80d-4247-b21c-112c023762d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.578364 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74906ee8-b80d-4247-b21c-112c023762d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.579242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74906ee8-b80d-4247-b21c-112c023762d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:45.581242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74906ee8-b80d-4247-b21c-112c023762d1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:45.609921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1322/2668] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:45.623139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1347/2669] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:45 2022] POST /identity/v3/projects => generated 369 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:45.623408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89ceff-f7cb-4bbe-a7bd-45e48d816472 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:45.623408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7f89ceff-f7cb-4bbe-a7bd-45e48d816472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:45.625128 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74906ee8-b80d-4247-b21c-112c023762d1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:45.702145 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89ceff-f7cb-4bbe-a7bd-45e48d816472 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:45.708707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74906ee8-b80d-4247-b21c-112c023762d1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:45.719871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6a910-6dd8-4154-b1b9-4b1ec3bc686c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:45.721101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6a910-6dd8-4154-b1b9-4b1ec3bc686c admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.721582 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6a910-6dd8-4154-b1b9-4b1ec3bc686c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.722113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6a910-6dd8-4154-b1b9-4b1ec3bc686c admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.723177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6a910-6dd8-4154-b1b9-4b1ec3bc686c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:45.726976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6a910-6dd8-4154-b1b9-4b1ec3bc686c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:45.755496 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:45.763971 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1348/2670] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:45 2022] POST /identity/v3/users => generated 285 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:45.768290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6a910-6dd8-4154-b1b9-4b1ec3bc686c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:45.778987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68f427-11d1-473c-be31-64ba6a6c83a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:45.779645 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68f427-11d1-473c-be31-64ba6a6c83a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.779956 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68f427-11d1-473c-be31-64ba6a6c83a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.780232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68f427-11d1-473c-be31-64ba6a6c83a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.782421 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-cb68f427-11d1-473c-be31-64ba6a6c83a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:45.784242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6a910-6dd8-4154-b1b9-4b1ec3bc686c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:45.791375 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c33c31-5a08-42a8-8b0f-353bc4598c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:45.792330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c33c31-5a08-42a8-8b0f-353bc4598c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.792730 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c33c31-5a08-42a8-8b0f-353bc4598c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.793124 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c33c31-5a08-42a8-8b0f-353bc4598c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.794257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c33c31-5a08-42a8-8b0f-353bc4598c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:45.797002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c33c31-5a08-42a8-8b0f-353bc4598c2b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:45.818185 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1349/2671] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:45.824641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c33c31-5a08-42a8-8b0f-353bc4598c2b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:45.842828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c33c31-5a08-42a8-8b0f-353bc4598c2b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:45.851502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704a562-a695-40fb-b63b-7e1b2e7b213d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:45.852959 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704a562-a695-40fb-b63b-7e1b2e7b213d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.853451 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704a562-a695-40fb-b63b-7e1b2e7b213d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.853932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704a562-a695-40fb-b63b-7e1b2e7b213d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.855117 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704a562-a695-40fb-b63b-7e1b2e7b213d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:45.858173 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704a562-a695-40fb-b63b-7e1b2e7b213d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:45.862336 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68f427-11d1-473c-be31-64ba6a6c83a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:45.865075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68f427-11d1-473c-be31-64ba6a6c83a0 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:45.876013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1350/2672] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:45.879588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1323/2673] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:45.880694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704a562-a695-40fb-b63b-7e1b2e7b213d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:45.891635 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704a562-a695-40fb-b63b-7e1b2e7b213d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:45.896590 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6cbb5-1728-43cd-a48a-09804fd28132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:45.897509 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6cbb5-1728-43cd-a48a-09804fd28132 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.897787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6cbb5-1728-43cd-a48a-09804fd28132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.898056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6cbb5-1728-43cd-a48a-09804fd28132 admin admin] PATH_INFO: `/v3/projects/5ad57bae6f1142f9a2bdb004a9e09958/users/3c6d88e1a2a24136b1523edd62b2f776/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:45.916270 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6cbb5-1728-43cd-a48a-09804fd28132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad57bae6f1142f9a2bdb004a9e09958, user_id=3c6d88e1a2a24136b1523edd62b2f776, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:45.918951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6cbb5-1728-43cd-a48a-09804fd28132 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:45.942501 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1351/2674] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:45 2022] PUT /identity/v3/projects/5ad57bae6f1142f9a2bdb004a9e09958/users/3c6d88e1a2a24136b1523edd62b2f776/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:45.961732 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aa660-5a10-441c-a2f8-e3972e7414ac None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:45.962273 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aa660-5a10-441c-a2f8-e3972e7414ac None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:45.962614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8aa660-5a10-441c-a2f8-e3972e7414ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:46.000556 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-3b8aa660-5a10-441c-a2f8-e3972e7414ac None None] MFA Rules not processed for user `3c6d88e1a2a24136b1523edd62b2f776`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:46.001882 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3b8aa660-5a10-441c-a2f8-e3972e7414ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:46.095531 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1324/2675] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:46.100318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6cbb5-1728-43cd-a48a-09804fd28132 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:46.203328 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6cbb5-1728-43cd-a48a-09804fd28132 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:46.209170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28ae44-2f0f-47b6-b076-4f4220eeddb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:46.210029 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28ae44-2f0f-47b6-b076-4f4220eeddb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:46.210527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28ae44-2f0f-47b6-b076-4f4220eeddb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:46.211032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28ae44-2f0f-47b6-b076-4f4220eeddb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:46.212159 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28ae44-2f0f-47b6-b076-4f4220eeddb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:46.214284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28ae44-2f0f-47b6-b076-4f4220eeddb7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:46.227122 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:46.253834 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1352/2676] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:46 2022] POST /identity/v3/projects => generated 367 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:46.260229 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aa660-5a10-441c-a2f8-e3972e7414ac None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:46.276970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8aa660-5a10-441c-a2f8-e3972e7414ac None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:46.386452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45701c41-950c-4926-9bd7-162599635cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:46.387697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45701c41-950c-4926-9bd7-162599635cfc admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:46.388211 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45701c41-950c-4926-9bd7-162599635cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:46.388696 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45701c41-950c-4926-9bd7-162599635cfc admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:46.389867 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45701c41-950c-4926-9bd7-162599635cfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:46.393793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45701c41-950c-4926-9bd7-162599635cfc admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:46.441110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1325/2677] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:46 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:46.448023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28ae44-2f0f-47b6-b076-4f4220eeddb7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:46.464780 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28ae44-2f0f-47b6-b076-4f4220eeddb7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:46.474895 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ad673-970f-4d9e-a213-bf72844215d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:46.475669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ad673-970f-4d9e-a213-bf72844215d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:46.475942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ad673-970f-4d9e-a213-bf72844215d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:46.476211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ad673-970f-4d9e-a213-bf72844215d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:46.476889 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ad673-970f-4d9e-a213-bf72844215d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:46.478541 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ad673-970f-4d9e-a213-bf72844215d6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:46.492796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1353/2678] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:46.497668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45701c41-950c-4926-9bd7-162599635cfc admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:46.509210 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45701c41-950c-4926-9bd7-162599635cfc admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:46.514086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f4b655-b163-4690-bbc4-0feb16c452a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:46.514770 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4b655-b163-4690-bbc4-0feb16c452a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:46.515043 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4b655-b163-4690-bbc4-0feb16c452a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:46.515311 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4b655-b163-4690-bbc4-0feb16c452a9 admin admin] PATH_INFO: `/v3/projects/4974c6358ba04612bd8d39b01e32c871/users/ad39cd812b4f42ec92f9b04bfb8b0b8b/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:46.525360 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:46.534722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f4b655-b163-4690-bbc4-0feb16c452a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4974c6358ba04612bd8d39b01e32c871, user_id=ad39cd812b4f42ec92f9b04bfb8b0b8b, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:46.537857 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f4b655-b163-4690-bbc4-0feb16c452a9 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:46.559586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1326/2679] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:46 2022] PUT /identity/v3/projects/4974c6358ba04612bd8d39b01e32c871/users/ad39cd812b4f42ec92f9b04bfb8b0b8b/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:46.565705 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ad673-970f-4d9e-a213-bf72844215d6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:46.580004 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ad673-970f-4d9e-a213-bf72844215d6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:46.659580 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7261e17-3056-406d-a279-f42ab254805a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:46.660632 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7261e17-3056-406d-a279-f42ab254805a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:46.661109 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7261e17-3056-406d-a279-f42ab254805a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:46.661568 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7261e17-3056-406d-a279-f42ab254805a admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:46.662655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7261e17-3056-406d-a279-f42ab254805a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:46.665557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7261e17-3056-406d-a279-f42ab254805a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:46.682585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1354/2680] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:46.687408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f4b655-b163-4690-bbc4-0feb16c452a9 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:46.701107 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f4b655-b163-4690-bbc4-0feb16c452a9 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:46.706185 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e3833-d304-42a3-ba7f-7a7a1e3401be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:46.707366 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e3833-d304-42a3-ba7f-7a7a1e3401be admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:46.707858 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e3833-d304-42a3-ba7f-7a7a1e3401be admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:46.708267 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e3833-d304-42a3-ba7f-7a7a1e3401be admin admin] PATH_INFO: `/v3/projects/4974c6358ba04612bd8d39b01e32c871/users/ad39cd812b4f42ec92f9b04bfb8b0b8b/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:46.710603 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e3833-d304-42a3-ba7f-7a7a1e3401be admin admin] RBAC: Authorizing `identity:create_grant(project_id=4974c6358ba04612bd8d39b01e32c871, user_id=ad39cd812b4f42ec92f9b04bfb8b0b8b, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:46.713638 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e3833-d304-42a3-ba7f-7a7a1e3401be admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:46.725629 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1327/2681] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:46 2022] PUT /identity/v3/projects/4974c6358ba04612bd8d39b01e32c871/users/ad39cd812b4f42ec92f9b04bfb8b0b8b/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:46.737871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69632b93-a978-4804-a59d-bae80618a813 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:46.738233 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69632b93-a978-4804-a59d-bae80618a813 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:46.738585 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69632b93-a978-4804-a59d-bae80618a813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:46.781408 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-69632b93-a978-4804-a59d-bae80618a813 None None] MFA Rules not processed for user `ad39cd812b4f42ec92f9b04bfb8b0b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:46.783418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-69632b93-a978-4804-a59d-bae80618a813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:46.886394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1355/2682] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:46.898527 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e3833-d304-42a3-ba7f-7a7a1e3401be admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:46.901232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6c9e3833-d304-42a3-ba7f-7a7a1e3401be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:46.978071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e3833-d304-42a3-ba7f-7a7a1e3401be admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:46.981986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfdc9f7-fc62-4d92-a865-ad391b164f0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:46.982621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdc9f7-fc62-4d92-a865-ad391b164f0a service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:46.982894 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdc9f7-fc62-4d92-a865-ad391b164f0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:46.983156 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdc9f7-fc62-4d92-a865-ad391b164f0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:46.993821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfdc9f7-fc62-4d92-a865-ad391b164f0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:46.995293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfdc9f7-fc62-4d92-a865-ad391b164f0a service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:47.007000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1328/2683] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:49.127439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69632b93-a978-4804-a59d-bae80618a813 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:49.273038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69632b93-a978-4804-a59d-bae80618a813 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:49.278372 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cc1e1-2726-41da-8a5a-5151616f839a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:49.279444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cc1e1-2726-41da-8a5a-5151616f839a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:49.279581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cc1e1-2726-41da-8a5a-5151616f839a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:49.279980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cc1e1-2726-41da-8a5a-5151616f839a admin admin] PATH_INFO: `/v3/users/3c6d88e1a2a24136b1523edd62b2f776` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:49.281245 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1cc1e1-2726-41da-8a5a-5151616f839a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c6d88e1a2a24136b1523edd62b2f776)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:49.283406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1cc1e1-2726-41da-8a5a-5151616f839a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:49.285690 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:49.367343 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-db1cc1e1-2726-41da-8a5a-5151616f839a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c6d88e1a2a24136b1523edd62b2f776'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:49.381143 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-db1cc1e1-2726-41da-8a5a-5151616f839a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c6d88e1a2a24136b1523edd62b2f776'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:49.382493 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-db1cc1e1-2726-41da-8a5a-5151616f839a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c6d88e1a2a24136b1523edd62b2f776'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:49.417929 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-db1cc1e1-2726-41da-8a5a-5151616f839a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c6d88e1a2a24136b1523edd62b2f776'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:49.428894 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-db1cc1e1-2726-41da-8a5a-5151616f839a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c6d88e1a2a24136b1523edd62b2f776'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:49.439756 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1356/2684] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:49 2022] DELETE /identity/v3/users/3c6d88e1a2a24136b1523edd62b2f776 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:49.448027 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ee768-8b12-4e9a-8b7a-65fb063330be None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:49.449309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ee768-8b12-4e9a-8b7a-65fb063330be None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:49.450510 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ee768-8b12-4e9a-8b7a-65fb063330be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:49.479365 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-dc6ee768-8b12-4e9a-8b7a-65fb063330be None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:49.480681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-dc6ee768-8b12-4e9a-8b7a-65fb063330be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:49.580656 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1329/2685] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:49.600242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cc1e1-2726-41da-8a5a-5151616f839a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:49.600242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-db1cc1e1-2726-41da-8a5a-5151616f839a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:49.710561 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cc1e1-2726-41da-8a5a-5151616f839a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:49.716058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076c7ee-8daa-476d-866d-c5d54daadf55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:49.716729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076c7ee-8daa-476d-866d-c5d54daadf55 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:49.717016 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076c7ee-8daa-476d-866d-c5d54daadf55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:49.717283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076c7ee-8daa-476d-866d-c5d54daadf55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:49.729932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076c7ee-8daa-476d-866d-c5d54daadf55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:49.731401 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076c7ee-8daa-476d-866d-c5d54daadf55 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:49.742921 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1357/2686] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:50.115264 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ee768-8b12-4e9a-8b7a-65fb063330be None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:50.117658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-dc6ee768-8b12-4e9a-8b7a-65fb063330be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:50.137039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ee768-8b12-4e9a-8b7a-65fb063330be None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:50.144428 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:50.145456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:50.145922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:50.146399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] PATH_INFO: `/v3/projects/5ad57bae6f1142f9a2bdb004a9e09958` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:50.150796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad57bae6f1142f9a2bdb004a9e09958)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:50.158699 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:50.224999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad57bae6f1142f9a2bdb004a9e09958', 'request_id': 'req-cdbfa562-c701-43a3-af3c-32ffab66ddd7'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.235063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad57bae6f1142f9a2bdb004a9e09958', 'request_id': 'req-cdbfa562-c701-43a3-af3c-32ffab66ddd7'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.242698 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad57bae6f1142f9a2bdb004a9e09958', 'request_id': 'req-cdbfa562-c701-43a3-af3c-32ffab66ddd7'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.254153 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:50.258816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] The token cache is being invalidate because project 5ad57bae6f1142f9a2bdb004a9e09958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:50.259726 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.261739 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.263680 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1330/2687] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:50 2022] DELETE /identity/v3/projects/5ad57bae6f1142f9a2bdb004a9e09958 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:50.267818 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076c7ee-8daa-476d-866d-c5d54daadf55 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:50.282167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076c7ee-8daa-476d-866d-c5d54daadf55 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:50.380662 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb08615-7ed1-4636-a361-3ef5064841d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:50.381176 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb08615-7ed1-4636-a361-3ef5064841d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:50.381493 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb08615-7ed1-4636-a361-3ef5064841d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:50.381798 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb08615-7ed1-4636-a361-3ef5064841d0 admin admin] PATH_INFO: `/v3/users/ad39cd812b4f42ec92f9b04bfb8b0b8b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:50.383971 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb08615-7ed1-4636-a361-3ef5064841d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad39cd812b4f42ec92f9b04bfb8b0b8b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:50.387148 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb08615-7ed1-4636-a361-3ef5064841d0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:50.424877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:50.460372 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5eb08615-7ed1-4636-a361-3ef5064841d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad39cd812b4f42ec92f9b04bfb8b0b8b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.471053 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5eb08615-7ed1-4636-a361-3ef5064841d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad39cd812b4f42ec92f9b04bfb8b0b8b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.472595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5eb08615-7ed1-4636-a361-3ef5064841d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad39cd812b4f42ec92f9b04bfb8b0b8b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.495240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5eb08615-7ed1-4636-a361-3ef5064841d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad39cd812b4f42ec92f9b04bfb8b0b8b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.501884 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:50.504186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5eb08615-7ed1-4636-a361-3ef5064841d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad39cd812b4f42ec92f9b04bfb8b0b8b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.505135 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:50.511250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1358/2688] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:50 2022] DELETE /identity/v3/users/ad39cd812b4f42ec92f9b04bfb8b0b8b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:50.526771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfa562-c701-43a3-af3c-32ffab66ddd7 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:50.600311 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe63e3b-2ea8-485f-9a9e-8eee4fef0703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:50.601489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe63e3b-2ea8-485f-9a9e-8eee4fef0703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:50.601980 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe63e3b-2ea8-485f-9a9e-8eee4fef0703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:50.602481 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe63e3b-2ea8-485f-9a9e-8eee4fef0703 admin admin] PATH_INFO: `/v3/users/eaec298d090444e9b737c1036ce8f31b` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:50.604589 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe63e3b-2ea8-485f-9a9e-8eee4fef0703 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaec298d090444e9b737c1036ce8f31b)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:50.608081 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe63e3b-2ea8-485f-9a9e-8eee4fef0703 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:50.689329 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bbe63e3b-2ea8-485f-9a9e-8eee4fef0703 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaec298d090444e9b737c1036ce8f31b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.710631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bbe63e3b-2ea8-485f-9a9e-8eee4fef0703 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaec298d090444e9b737c1036ce8f31b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.728792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bbe63e3b-2ea8-485f-9a9e-8eee4fef0703 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaec298d090444e9b737c1036ce8f31b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.755397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bbe63e3b-2ea8-485f-9a9e-8eee4fef0703 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaec298d090444e9b737c1036ce8f31b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.755397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-bbe63e3b-2ea8-485f-9a9e-8eee4fef0703 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaec298d090444e9b737c1036ce8f31b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:50.768565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1331/2689] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:50 2022] DELETE /identity/v3/users/eaec298d090444e9b737c1036ce8f31b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:50.774116 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bd6fd-bbad-46a7-a0fe-63c48070411a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:50.774912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bd6fd-bbad-46a7-a0fe-63c48070411a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:50.776546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bd6fd-bbad-46a7-a0fe-63c48070411a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:50.814870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2a9bd6fd-bbad-46a7-a0fe-63c48070411a None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:50.816439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2a9bd6fd-bbad-46a7-a0fe-63c48070411a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:50.892831 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe63e3b-2ea8-485f-9a9e-8eee4fef0703 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:50.895012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-bbe63e3b-2ea8-485f-9a9e-8eee4fef0703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:50.920884 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1359/2690] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:50.925434 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bd6fd-bbad-46a7-a0fe-63c48070411a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:50.925434 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2a9bd6fd-bbad-46a7-a0fe-63c48070411a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:51.007222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bd6fd-bbad-46a7-a0fe-63c48070411a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:51.009977 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe63e3b-2ea8-485f-9a9e-8eee4fef0703 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:51.011919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4232a-c675-473c-83c0-074cdcfb877a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:51.012554 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4232a-c675-473c-83c0-074cdcfb877a service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.012839 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4232a-c675-473c-83c0-074cdcfb877a service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.013102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4232a-c675-473c-83c0-074cdcfb877a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.014948 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e0e4232a-c675-473c-83c0-074cdcfb877a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:51.017565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:51.018501 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.018849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.019306 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85 admin admin] PATH_INFO: `/v3/projects/4974c6358ba04612bd8d39b01e32c871` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.021408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4974c6358ba04612bd8d39b01e32c871)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:51.025105 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:51.062194 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:51.092225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:51.099709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4974c6358ba04612bd8d39b01e32c871', 'request_id': 'req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:51.109990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4974c6358ba04612bd8d39b01e32c871', 'request_id': 'req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:51.111065 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4232a-c675-473c-83c0-074cdcfb877a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:51.113738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4232a-c675-473c-83c0-074cdcfb877a service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:51.118899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4974c6358ba04612bd8d39b01e32c871', 'request_id': 'req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:51.127531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1360/2691] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:51.129421 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4232a-c675-473c-83c0-074cdcfb877a service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:51.132149 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e0e4232a-c675-473c-83c0-074cdcfb877a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:51.136948 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85 admin admin] The token cache is being invalidate because project 4974c6358ba04612bd8d39b01e32c871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:51.137252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:51.138335 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-2aa439ce-bd1a-4cd8-973e-e83546f0ed85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:51.141416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1332/2692] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:50 2022] DELETE /identity/v3/projects/4974c6358ba04612bd8d39b01e32c871 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:51.149280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1c53e-b44d-41a2-94e1-87299e26225e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.149654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1c53e-b44d-41a2-94e1-87299e26225e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.150091 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1c53e-b44d-41a2-94e1-87299e26225e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.183649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-b3b1c53e-b44d-41a2-94e1-87299e26225e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:51.185954 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b3b1c53e-b44d-41a2-94e1-87299e26225e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:51.233855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4232a-c675-473c-83c0-074cdcfb877a service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:51.312802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1333/2693] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:51.317700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f678d02-b9d5-4df1-b139-f101767ad22c None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.318118 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f678d02-b9d5-4df1-b139-f101767ad22c None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.318506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f678d02-b9d5-4df1-b139-f101767ad22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.326809 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14d1af-9f44-41a3-9c36-0e3b249898be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:51.327869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14d1af-9f44-41a3-9c36-0e3b249898be service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.328427 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14d1af-9f44-41a3-9c36-0e3b249898be service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.328955 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14d1af-9f44-41a3-9c36-0e3b249898be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.331698 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0d14d1af-9f44-41a3-9c36-0e3b249898be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:51.351419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14d1af-9f44-41a3-9c36-0e3b249898be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:51.353770 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-0f678d02-b9d5-4df1-b139-f101767ad22c None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:51.354184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14d1af-9f44-41a3-9c36-0e3b249898be service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:51.356092 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0f678d02-b9d5-4df1-b139-f101767ad22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:51.369222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1361/2694] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:51.370414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1334/2695] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:51.377403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f678d02-b9d5-4df1-b139-f101767ad22c None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:51.395113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f678d02-b9d5-4df1-b139-f101767ad22c None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:51.402613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d2a26-5093-4cf6-993c-4e54e19e192d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:51.402896 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d2a26-5093-4cf6-993c-4e54e19e192d admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.403246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d2a26-5093-4cf6-993c-4e54e19e192d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.403547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d2a26-5093-4cf6-993c-4e54e19e192d admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.404913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d2a26-5093-4cf6-993c-4e54e19e192d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:51.405493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d2a26-5093-4cf6-993c-4e54e19e192d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:51.408547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d2a26-5093-4cf6-993c-4e54e19e192d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:51.432457 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1335/2696] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:43:51.439292 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14d1af-9f44-41a3-9c36-0e3b249898be service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:51.460189 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14d1af-9f44-41a3-9c36-0e3b249898be service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:51.468332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65867870-102d-444c-8238-dc19e7a5855a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:51.471344 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65867870-102d-444c-8238-dc19e7a5855a admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.472018 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65867870-102d-444c-8238-dc19e7a5855a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.473557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65867870-102d-444c-8238-dc19e7a5855a admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.475026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65867870-102d-444c-8238-dc19e7a5855a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:51.479118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65867870-102d-444c-8238-dc19e7a5855a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:51.522403 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1362/2697] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:51 2022] POST /identity/v3/projects => generated 387 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:51.528688 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d2a26-5093-4cf6-993c-4e54e19e192d admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:51.550137 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d2a26-5093-4cf6-993c-4e54e19e192d admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:51.651481 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061190b6-fc63-435b-aa24-6f93b92dc46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:51.652134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061190b6-fc63-435b-aa24-6f93b92dc46f admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.652445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061190b6-fc63-435b-aa24-6f93b92dc46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.652721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061190b6-fc63-435b-aa24-6f93b92dc46f admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.653298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061190b6-fc63-435b-aa24-6f93b92dc46f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:51.655232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061190b6-fc63-435b-aa24-6f93b92dc46f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:51.685149 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1336/2698] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:51 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:51.690897 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65867870-102d-444c-8238-dc19e7a5855a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:51.709610 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65867870-102d-444c-8238-dc19e7a5855a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:51.715954 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8977a4-728d-4b6f-93dc-8c19b19ba637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:51.716576 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8977a4-728d-4b6f-93dc-8c19b19ba637 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.716847 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8977a4-728d-4b6f-93dc-8c19b19ba637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.717110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8977a4-728d-4b6f-93dc-8c19b19ba637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.717769 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8977a4-728d-4b6f-93dc-8c19b19ba637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:51.720306 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8977a4-728d-4b6f-93dc-8c19b19ba637 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:51.751048 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1363/2699] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:51.753803 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061190b6-fc63-435b-aa24-6f93b92dc46f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:51.771677 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061190b6-fc63-435b-aa24-6f93b92dc46f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:51.777149 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0758dc-45c0-4139-b167-267ad9c6108c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:51.777794 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0758dc-45c0-4139-b167-267ad9c6108c admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.778247 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0758dc-45c0-4139-b167-267ad9c6108c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.778414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0758dc-45c0-4139-b167-267ad9c6108c admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.779102 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0758dc-45c0-4139-b167-267ad9c6108c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:51.780756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0758dc-45c0-4139-b167-267ad9c6108c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:51.798534 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1337/2700] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:51.805363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0758dc-45c0-4139-b167-267ad9c6108c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:51.805738 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8977a4-728d-4b6f-93dc-8c19b19ba637 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:51.818256 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-4c0758dc-45c0-4139-b167-267ad9c6108c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:51.826055 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8977a4-728d-4b6f-93dc-8c19b19ba637 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:51.833384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fab2d0-c84a-4034-b495-31097ceb2e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:51.834221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fab2d0-c84a-4034-b495-31097ceb2e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.834575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fab2d0-c84a-4034-b495-31097ceb2e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.834781 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fab2d0-c84a-4034-b495-31097ceb2e54 admin admin] PATH_INFO: `/v3/projects/978e1deb943348739dd394ab2f54bfba/users/6c4eb35cdca144339019af79fd63604b/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.837485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0758dc-45c0-4139-b167-267ad9c6108c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:51.849597 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:51.851059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.852764 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.852764 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] PATH_INFO: `/v3/projects/78110234c61d4064bddb827f6ff4220d` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.856020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] RBAC: Authorizing `identity:delete_project(project_id=78110234c61d4064bddb827f6ff4220d)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:51.859591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:51.861366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fab2d0-c84a-4034-b495-31097ceb2e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=978e1deb943348739dd394ab2f54bfba, user_id=6c4eb35cdca144339019af79fd63604b, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:51.864077 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fab2d0-c84a-4034-b495-31097ceb2e54 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:51.864498 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:51.883733 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1364/2701] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:51 2022] PUT /identity/v3/projects/978e1deb943348739dd394ab2f54bfba/users/6c4eb35cdca144339019af79fd63604b/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:51.897843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abe0f1-3d51-46fc-9853-a4524174af04 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:51.898112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abe0f1-3d51-46fc-9853-a4524174af04 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:51.898914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abe0f1-3d51-46fc-9853-a4524174af04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:51.942602 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-64abe0f1-3d51-46fc-9853-a4524174af04 None None] MFA Rules not processed for user `6c4eb35cdca144339019af79fd63604b`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:51.944385 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-64abe0f1-3d51-46fc-9853-a4524174af04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:51.946547 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78110234c61d4064bddb827f6ff4220d', 'request_id': 'req-83c8ec33-20d5-4741-9b05-8eef5668d20a'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:51.958569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78110234c61d4064bddb827f6ff4220d', 'request_id': 'req-83c8ec33-20d5-4741-9b05-8eef5668d20a'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:51.968422 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78110234c61d4064bddb827f6ff4220d', 'request_id': 'req-83c8ec33-20d5-4741-9b05-8eef5668d20a'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:51.988323 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] The token cache is being invalidate because project 78110234c61d4064bddb827f6ff4220d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:51.988744 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:51.989688 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:51.991565 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1338/2702] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:51 2022] DELETE /identity/v3/projects/78110234c61d4064bddb827f6ff4220d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:51.996864 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:51.998929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:52.009608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:52.047662 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1365/2703] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:52.055296 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abe0f1-3d51-46fc-9853-a4524174af04 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:52.057574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-64abe0f1-3d51-46fc-9853-a4524174af04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:52.087979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8ec33-20d5-4741-9b05-8eef5668d20a admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:52.093629 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec360d60-104c-4293-bbe5-50e5c492bab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:52.094322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec360d60-104c-4293-bbe5-50e5c492bab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:52.094599 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec360d60-104c-4293-bbe5-50e5c492bab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:52.094872 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec360d60-104c-4293-bbe5-50e5c492bab3 admin admin] PATH_INFO: `/v3/users/5dd8cfde2663410caca67c2d4d2103fc` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:52.096135 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec360d60-104c-4293-bbe5-50e5c492bab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dd8cfde2663410caca67c2d4d2103fc)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:52.098113 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec360d60-104c-4293-bbe5-50e5c492bab3 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:52.159347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ec360d60-104c-4293-bbe5-50e5c492bab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dd8cfde2663410caca67c2d4d2103fc'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.164346 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abe0f1-3d51-46fc-9853-a4524174af04 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:52.170431 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256fab8-770e-464d-9dd8-027fdac7e06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:52.171161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256fab8-770e-464d-9dd8-027fdac7e06b admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:52.171440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256fab8-770e-464d-9dd8-027fdac7e06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:52.171709 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256fab8-770e-464d-9dd8-027fdac7e06b admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:52.172425 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256fab8-770e-464d-9dd8-027fdac7e06b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:52.174500 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256fab8-770e-464d-9dd8-027fdac7e06b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:52.180259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ec360d60-104c-4293-bbe5-50e5c492bab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dd8cfde2663410caca67c2d4d2103fc'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.191366 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ec360d60-104c-4293-bbe5-50e5c492bab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dd8cfde2663410caca67c2d4d2103fc'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.202932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ec360d60-104c-4293-bbe5-50e5c492bab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dd8cfde2663410caca67c2d4d2103fc'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.203700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ec360d60-104c-4293-bbe5-50e5c492bab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dd8cfde2663410caca67c2d4d2103fc'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.213587 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1339/2704] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:51 2022] DELETE /identity/v3/users/5dd8cfde2663410caca67c2d4d2103fc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:52.218104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1366/2705] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:52 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:52.222204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec360d60-104c-4293-bbe5-50e5c492bab3 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:52.223993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ec360d60-104c-4293-bbe5-50e5c492bab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:52.238851 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:52.328883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec360d60-104c-4293-bbe5-50e5c492bab3 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:52.404170 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c4b46-fd80-483e-a8d8-6a9fdb248859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:52.404807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c4b46-fd80-483e-a8d8-6a9fdb248859 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:52.405321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c4b46-fd80-483e-a8d8-6a9fdb248859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:52.405614 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c4b46-fd80-483e-a8d8-6a9fdb248859 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:52.406363 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c4b46-fd80-483e-a8d8-6a9fdb248859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:52.408743 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c4b46-fd80-483e-a8d8-6a9fdb248859 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:52.442534 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1340/2706] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:52 2022] POST /identity/v3/users => generated 300 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:52.451543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256fab8-770e-464d-9dd8-027fdac7e06b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:52.469043 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256fab8-770e-464d-9dd8-027fdac7e06b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:52.477112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d15bfd-fa3b-4ad9-8db3-50accc449e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:52.478417 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d15bfd-fa3b-4ad9-8db3-50accc449e4d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:52.478767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d15bfd-fa3b-4ad9-8db3-50accc449e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:52.479117 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d15bfd-fa3b-4ad9-8db3-50accc449e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:52.480048 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d15bfd-fa3b-4ad9-8db3-50accc449e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:52.482730 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d15bfd-fa3b-4ad9-8db3-50accc449e4d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:52.495694 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1367/2707] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:52.501276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c4b46-fd80-483e-a8d8-6a9fdb248859 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:52.515637 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c4b46-fd80-483e-a8d8-6a9fdb248859 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:52.521709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716e54a-4f13-46fd-bb13-f00b5e379eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:52.522557 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716e54a-4f13-46fd-bb13-f00b5e379eae admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:52.522979 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716e54a-4f13-46fd-bb13-f00b5e379eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:52.523461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716e54a-4f13-46fd-bb13-f00b5e379eae admin admin] PATH_INFO: `/v3/projects/5468be04256b408f8910fe5e56ea52bd/users/55dc125b79dd4630917687de4936644b/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:52.540893 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716e54a-4f13-46fd-bb13-f00b5e379eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=5468be04256b408f8910fe5e56ea52bd, user_id=55dc125b79dd4630917687de4936644b, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:52.543641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716e54a-4f13-46fd-bb13-f00b5e379eae admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:52.560700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1341/2708] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:52 2022] PUT /identity/v3/projects/5468be04256b408f8910fe5e56ea52bd/users/55dc125b79dd4630917687de4936644b/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:52.564844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d15bfd-fa3b-4ad9-8db3-50accc449e4d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:52.582785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d15bfd-fa3b-4ad9-8db3-50accc449e4d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:52.637261 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716e54a-4f13-46fd-bb13-f00b5e379eae admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:52.640188 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b716e54a-4f13-46fd-bb13-f00b5e379eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:52.702059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a6210-24c5-4953-aba3-4e786afe11f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:52.703008 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a6210-24c5-4953-aba3-4e786afe11f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:52.703478 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a6210-24c5-4953-aba3-4e786afe11f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:52.703669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a6210-24c5-4953-aba3-4e786afe11f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:52.704513 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a6210-24c5-4953-aba3-4e786afe11f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:52.706180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a6210-24c5-4953-aba3-4e786afe11f3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:52.718227 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1368/2709] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:52.722010 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a6210-24c5-4953-aba3-4e786afe11f3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:52.736357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a6210-24c5-4953-aba3-4e786afe11f3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:52.744683 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4cf36-0592-4e03-a480-b7f993cc0f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:52.745450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4cf36-0592-4e03-a480-b7f993cc0f46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:52.745786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4cf36-0592-4e03-a480-b7f993cc0f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:52.746111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4cf36-0592-4e03-a480-b7f993cc0f46 admin admin] PATH_INFO: `/v3/projects/5468be04256b408f8910fe5e56ea52bd/users/55dc125b79dd4630917687de4936644b/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:52.751942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4cf36-0592-4e03-a480-b7f993cc0f46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5468be04256b408f8910fe5e56ea52bd, user_id=55dc125b79dd4630917687de4936644b, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:52.754784 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4cf36-0592-4e03-a480-b7f993cc0f46 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:52.759848 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716e54a-4f13-46fd-bb13-f00b5e379eae admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:52.767072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2ac07-550c-4a93-a1c3-c43cd165908b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:52.768175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2ac07-550c-4a93-a1c3-c43cd165908b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:52.768602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2ac07-550c-4a93-a1c3-c43cd165908b admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:52.768879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2ac07-550c-4a93-a1c3-c43cd165908b admin admin] PATH_INFO: `/v3/projects/432b028d7e3b4440bd2a0564d6108e69` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:52.770801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2ac07-550c-4a93-a1c3-c43cd165908b admin admin] RBAC: Authorizing `identity:delete_project(project_id=432b028d7e3b4440bd2a0564d6108e69)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:52.772781 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2ac07-550c-4a93-a1c3-c43cd165908b admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:52.776520 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1369/2710] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:52 2022] PUT /identity/v3/projects/5468be04256b408f8910fe5e56ea52bd/users/55dc125b79dd4630917687de4936644b/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:52.794130 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d04f12-0684-4a64-9c27-11173491b489 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:52.794612 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d04f12-0684-4a64-9c27-11173491b489 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:52.795478 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d04f12-0684-4a64-9c27-11173491b489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:52.817802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:52.834669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-94d04f12-0684-4a64-9c27-11173491b489 None None] MFA Rules not processed for user `55dc125b79dd4630917687de4936644b`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:52.836262 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-94d04f12-0684-4a64-9c27-11173491b489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:52.843016 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-51c2ac07-550c-4a93-a1c3-c43cd165908b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '432b028d7e3b4440bd2a0564d6108e69', 'request_id': 'req-51c2ac07-550c-4a93-a1c3-c43cd165908b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.853987 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-51c2ac07-550c-4a93-a1c3-c43cd165908b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '432b028d7e3b4440bd2a0564d6108e69', 'request_id': 'req-51c2ac07-550c-4a93-a1c3-c43cd165908b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.863125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-51c2ac07-550c-4a93-a1c3-c43cd165908b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '432b028d7e3b4440bd2a0564d6108e69', 'request_id': 'req-51c2ac07-550c-4a93-a1c3-c43cd165908b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.885212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:52.888663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-51c2ac07-550c-4a93-a1c3-c43cd165908b admin admin] The token cache is being invalidate because project 432b028d7e3b4440bd2a0564d6108e69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:52.889124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-51c2ac07-550c-4a93-a1c3-c43cd165908b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.890117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-51c2ac07-550c-4a93-a1c3-c43cd165908b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:52.893356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1342/2711] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:52 2022] DELETE /identity/v3/projects/432b028d7e3b4440bd2a0564d6108e69 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:52.900403 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d328b-b812-464e-934e-81f4fb5fadaa None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:52.900863 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d328b-b812-464e-934e-81f4fb5fadaa None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:52.901314 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d328b-b812-464e-934e-81f4fb5fadaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:52.936143 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-187d328b-b812-464e-934e-81f4fb5fadaa None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:52.938172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-187d328b-b812-464e-934e-81f4fb5fadaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:52.944993 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1370/2712] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:52 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:52.961058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d04f12-0684-4a64-9c27-11173491b489 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:52.963186 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-94d04f12-0684-4a64-9c27-11173491b489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:53.039790 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1343/2713] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:53.052214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fda756-a786-400c-be53-d981ea9b4150 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.056078 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fda756-a786-400c-be53-d981ea9b4150 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.056360 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fda756-a786-400c-be53-d981ea9b4150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.063608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d04f12-0684-4a64-9c27-11173491b489 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.070288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06c5bb-7919-4b97-b856-0fdb2b42e279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.071444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06c5bb-7919-4b97-b856-0fdb2b42e279 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.071660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06c5bb-7919-4b97-b856-0fdb2b42e279 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.071958 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06c5bb-7919-4b97-b856-0fdb2b42e279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.081399 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-c2fda756-a786-400c-be53-d981ea9b4150 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:53.082784 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c2fda756-a786-400c-be53-d981ea9b4150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:53.086392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06c5bb-7919-4b97-b856-0fdb2b42e279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.087922 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06c5bb-7919-4b97-b856-0fdb2b42e279 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.095663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1344/2714] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:53.101122 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fda756-a786-400c-be53-d981ea9b4150 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.102235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1371/2715] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:53.114316 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fda756-a786-400c-be53-d981ea9b4150 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.124671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc23ce-ba41-453e-a73c-b96baa9a02e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.125297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc23ce-ba41-453e-a73c-b96baa9a02e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.125573 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc23ce-ba41-453e-a73c-b96baa9a02e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.125904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc23ce-ba41-453e-a73c-b96baa9a02e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.126507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc23ce-ba41-453e-a73c-b96baa9a02e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:53.126898 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc23ce-ba41-453e-a73c-b96baa9a02e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.129805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc23ce-ba41-453e-a73c-b96baa9a02e1 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.150715 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1345/2716] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:43:53.151464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06c5bb-7919-4b97-b856-0fdb2b42e279 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.164159 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06c5bb-7919-4b97-b856-0fdb2b42e279 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.169642 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e598565-3338-42e9-9288-89859ebddab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.170397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e598565-3338-42e9-9288-89859ebddab8 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.171072 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e598565-3338-42e9-9288-89859ebddab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.171226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e598565-3338-42e9-9288-89859ebddab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.172007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e598565-3338-42e9-9288-89859ebddab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.174965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e598565-3338-42e9-9288-89859ebddab8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.212701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1372/2717] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:53 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:53.218808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc23ce-ba41-453e-a73c-b96baa9a02e1 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.233263 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc23ce-ba41-453e-a73c-b96baa9a02e1 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.308923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694df0f-a0a6-41a3-8392-7218f3cfa4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.309922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694df0f-a0a6-41a3-8392-7218f3cfa4cf admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.310381 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694df0f-a0a6-41a3-8392-7218f3cfa4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.310823 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694df0f-a0a6-41a3-8392-7218f3cfa4cf admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.312749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694df0f-a0a6-41a3-8392-7218f3cfa4cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.314811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694df0f-a0a6-41a3-8392-7218f3cfa4cf admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.345985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1346/2718] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:53 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:53.350511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e598565-3338-42e9-9288-89859ebddab8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.362063 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e598565-3338-42e9-9288-89859ebddab8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.369497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9eb936-e2ab-40be-a506-5332f02f4129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.370525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9eb936-e2ab-40be-a506-5332f02f4129 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.371023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9eb936-e2ab-40be-a506-5332f02f4129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.371475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9eb936-e2ab-40be-a506-5332f02f4129 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.372686 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9eb936-e2ab-40be-a506-5332f02f4129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.375789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9eb936-e2ab-40be-a506-5332f02f4129 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.390708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1373/2719] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:53.395659 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694df0f-a0a6-41a3-8392-7218f3cfa4cf admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.411343 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694df0f-a0a6-41a3-8392-7218f3cfa4cf admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.418814 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac00946-3037-492f-8d25-5471cf614ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.420000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac00946-3037-492f-8d25-5471cf614ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.420351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac00946-3037-492f-8d25-5471cf614ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.420712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac00946-3037-492f-8d25-5471cf614ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.421489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac00946-3037-492f-8d25-5471cf614ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.423089 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac00946-3037-492f-8d25-5471cf614ca3 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.435613 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1347/2720] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:53.440445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9eb936-e2ab-40be-a506-5332f02f4129 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.454556 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9eb936-e2ab-40be-a506-5332f02f4129 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.460232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8d2cd-9fcc-4eae-a006-3a10622364db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.460950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8d2cd-9fcc-4eae-a006-3a10622364db admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.461244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8d2cd-9fcc-4eae-a006-3a10622364db admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.461516 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8d2cd-9fcc-4eae-a006-3a10622364db admin admin] PATH_INFO: `/v3/projects/c0ee0111943f427a8d7745468c651166/users/a20de45f4e9643f1bde1be7389a09b00/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.487168 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8d2cd-9fcc-4eae-a006-3a10622364db admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ee0111943f427a8d7745468c651166, user_id=a20de45f4e9643f1bde1be7389a09b00, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.489973 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8d2cd-9fcc-4eae-a006-3a10622364db admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.507159 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1374/2721] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:53 2022] PUT /identity/v3/projects/c0ee0111943f427a8d7745468c651166/users/a20de45f4e9643f1bde1be7389a09b00/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:53.523881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d790440-9f93-4636-a2fb-dc234b0d565c None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.524576 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d790440-9f93-4636-a2fb-dc234b0d565c None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.525079 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d790440-9f93-4636-a2fb-dc234b0d565c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.556847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:53.563063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-9d790440-9f93-4636-a2fb-dc234b0d565c None None] MFA Rules not processed for user `a20de45f4e9643f1bde1be7389a09b00`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:53.564686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9d790440-9f93-4636-a2fb-dc234b0d565c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:53.663630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1348/2722] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:53.675177 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8d2cd-9fcc-4eae-a006-3a10622364db admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.678374 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b9c8d2cd-9fcc-4eae-a006-3a10622364db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:53.761860 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8d2cd-9fcc-4eae-a006-3a10622364db admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.768801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bdf255-bd7c-41c9-9879-92943c5b6225 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.769836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdf255-bd7c-41c9-9879-92943c5b6225 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.770144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdf255-bd7c-41c9-9879-92943c5b6225 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.770435 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdf255-bd7c-41c9-9879-92943c5b6225 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.791584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bdf255-bd7c-41c9-9879-92943c5b6225 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.793519 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bdf255-bd7c-41c9-9879-92943c5b6225 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:53.806221 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1375/2723] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:53.871587 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d790440-9f93-4636-a2fb-dc234b0d565c None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:53.873365 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9d790440-9f93-4636-a2fb-dc234b0d565c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:53.960385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d790440-9f93-4636-a2fb-dc234b0d565c None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:53.967569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c501b-ef6c-42b1-b004-160a5db7e925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:53.968780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c501b-ef6c-42b1-b004-160a5db7e925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:53.968780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c501b-ef6c-42b1-b004-160a5db7e925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:53.968780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c501b-ef6c-42b1-b004-160a5db7e925 admin admin] PATH_INFO: `/v3/users/6c4eb35cdca144339019af79fd63604b` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:53.970034 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c501b-ef6c-42b1-b004-160a5db7e925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c4eb35cdca144339019af79fd63604b)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:53.972031 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c501b-ef6c-42b1-b004-160a5db7e925 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:54.033706 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ef5c501b-ef6c-42b1-b004-160a5db7e925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c4eb35cdca144339019af79fd63604b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:54.048599 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ef5c501b-ef6c-42b1-b004-160a5db7e925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c4eb35cdca144339019af79fd63604b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:54.058163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ef5c501b-ef6c-42b1-b004-160a5db7e925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c4eb35cdca144339019af79fd63604b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:54.072141 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ef5c501b-ef6c-42b1-b004-160a5db7e925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c4eb35cdca144339019af79fd63604b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:54.074759 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ef5c501b-ef6c-42b1-b004-160a5db7e925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c4eb35cdca144339019af79fd63604b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:54.085830 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1349/2724] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:53 2022] DELETE /identity/v3/users/6c4eb35cdca144339019af79fd63604b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:54.091572 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25add1e-863e-49f2-832c-daa4f8034a4c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.091863 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25add1e-863e-49f2-832c-daa4f8034a4c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.092175 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25add1e-863e-49f2-832c-daa4f8034a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.117890 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a25add1e-863e-49f2-832c-daa4f8034a4c None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:54.119331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a25add1e-863e-49f2-832c-daa4f8034a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:54.129704 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:54.266096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1376/2725] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:54.280501 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c501b-ef6c-42b1-b004-160a5db7e925 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:54.282385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ef5c501b-ef6c-42b1-b004-160a5db7e925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:54.315157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:54.363792 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c501b-ef6c-42b1-b004-160a5db7e925 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:54.370098 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752ba43-b4ec-4912-8e54-bf5fffef8dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:54.370859 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752ba43-b4ec-4912-8e54-bf5fffef8dfe service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.371125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752ba43-b4ec-4912-8e54-bf5fffef8dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.371392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752ba43-b4ec-4912-8e54-bf5fffef8dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.382358 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752ba43-b4ec-4912-8e54-bf5fffef8dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:54.384337 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752ba43-b4ec-4912-8e54-bf5fffef8dfe service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:54.395476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1350/2726] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:54.720141 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25add1e-863e-49f2-832c-daa4f8034a4c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:54.721982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a25add1e-863e-49f2-832c-daa4f8034a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:54.740366 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25add1e-863e-49f2-832c-daa4f8034a4c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:54.750972 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:54.752399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.759002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.759002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] PATH_INFO: `/v3/projects/978e1deb943348739dd394ab2f54bfba` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.759002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=978e1deb943348739dd394ab2f54bfba)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:54.762490 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:54.825861 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '978e1deb943348739dd394ab2f54bfba', 'request_id': 'req-266f3c17-37f0-4905-b326-732a593787e4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:54.832233 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '978e1deb943348739dd394ab2f54bfba', 'request_id': 'req-266f3c17-37f0-4905-b326-732a593787e4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:54.839317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '978e1deb943348739dd394ab2f54bfba', 'request_id': 'req-266f3c17-37f0-4905-b326-732a593787e4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:54.857670 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] The token cache is being invalidate because project 978e1deb943348739dd394ab2f54bfba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:54.857955 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:54.858659 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:54.860548 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1377/2727] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:54 2022] DELETE /identity/v3/projects/978e1deb943348739dd394ab2f54bfba => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:54.864580 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752ba43-b4ec-4912-8e54-bf5fffef8dfe service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:54.875652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752ba43-b4ec-4912-8e54-bf5fffef8dfe service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:54.950400 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e61ac4-f255-4ff5-b894-b293e2ef71f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:54.951081 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e61ac4-f255-4ff5-b894-b293e2ef71f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.951402 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e61ac4-f255-4ff5-b894-b293e2ef71f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.951663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e61ac4-f255-4ff5-b894-b293e2ef71f2 admin admin] PATH_INFO: `/v3/users/55dc125b79dd4630917687de4936644b` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.952865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e61ac4-f255-4ff5-b894-b293e2ef71f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55dc125b79dd4630917687de4936644b)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:54.954875 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e61ac4-f255-4ff5-b894-b293e2ef71f2 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:55.025096 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-87e61ac4-f255-4ff5-b894-b293e2ef71f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55dc125b79dd4630917687de4936644b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:55.040321 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-87e61ac4-f255-4ff5-b894-b293e2ef71f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55dc125b79dd4630917687de4936644b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:55.048422 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-87e61ac4-f255-4ff5-b894-b293e2ef71f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55dc125b79dd4630917687de4936644b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:55.062359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-87e61ac4-f255-4ff5-b894-b293e2ef71f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55dc125b79dd4630917687de4936644b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:55.063123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-87e61ac4-f255-4ff5-b894-b293e2ef71f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55dc125b79dd4630917687de4936644b'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:55.073492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1351/2728] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:54 2022] DELETE /identity/v3/users/55dc125b79dd4630917687de4936644b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:55.449870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:55.456113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:55.486532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f3c17-37f0-4905-b326-732a593787e4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:55.567722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514e91b-ac77-4370-9061-0af44644caa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:55.569061 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514e91b-ac77-4370-9061-0af44644caa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:55.569641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514e91b-ac77-4370-9061-0af44644caa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:55.570295 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514e91b-ac77-4370-9061-0af44644caa4 admin admin] PATH_INFO: `/v3/projects/5468be04256b408f8910fe5e56ea52bd` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:55.572871 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514e91b-ac77-4370-9061-0af44644caa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5468be04256b408f8910fe5e56ea52bd)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:55.575487 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514e91b-ac77-4370-9061-0af44644caa4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:55.638057 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:55.652989 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3514e91b-ac77-4370-9061-0af44644caa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5468be04256b408f8910fe5e56ea52bd', 'request_id': 'req-3514e91b-ac77-4370-9061-0af44644caa4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:55.663982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3514e91b-ac77-4370-9061-0af44644caa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5468be04256b408f8910fe5e56ea52bd', 'request_id': 'req-3514e91b-ac77-4370-9061-0af44644caa4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:55.671190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3514e91b-ac77-4370-9061-0af44644caa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5468be04256b408f8910fe5e56ea52bd', 'request_id': 'req-3514e91b-ac77-4370-9061-0af44644caa4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:55.689438 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3514e91b-ac77-4370-9061-0af44644caa4 admin admin] The token cache is being invalidate because project 5468be04256b408f8910fe5e56ea52bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:55.690056 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3514e91b-ac77-4370-9061-0af44644caa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:55.691099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3514e91b-ac77-4370-9061-0af44644caa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:55.693439 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1378/2729] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:55 2022] DELETE /identity/v3/projects/5468be04256b408f8910fe5e56ea52bd => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:55.701520 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f745868-b974-437b-abe4-58485df84487 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:55.701976 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f745868-b974-437b-abe4-58485df84487 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:55.702397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f745868-b974-437b-abe4-58485df84487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:55.734816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-9f745868-b974-437b-abe4-58485df84487 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:55.736626 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9f745868-b974-437b-abe4-58485df84487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:55.796310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:55.837689 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1352/2730] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:55.847350 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7328b-eb06-4c26-89c6-11ad3eb1481c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:55.849811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7328b-eb06-4c26-89c6-11ad3eb1481c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:55.849811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7328b-eb06-4c26-89c6-11ad3eb1481c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:55.878888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-94d7328b-eb06-4c26-89c6-11ad3eb1481c None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:55.880703 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-94d7328b-eb06-4c26-89c6-11ad3eb1481c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:55.890768 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1379/2731] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:55.895981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f745868-b974-437b-abe4-58485df84487 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:55.908865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f745868-b974-437b-abe4-58485df84487 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:55.915984 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb2c2b-7f88-452b-961c-90f419ff125c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:55.916496 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb2c2b-7f88-452b-961c-90f419ff125c admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:55.916770 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb2c2b-7f88-452b-961c-90f419ff125c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:55.917060 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb2c2b-7f88-452b-961c-90f419ff125c admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:55.917761 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb2c2b-7f88-452b-961c-90f419ff125c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:55.918067 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb2c2b-7f88-452b-961c-90f419ff125c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:55.919832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb2c2b-7f88-452b-961c-90f419ff125c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:55.936171 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1353/2732] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:55.940933 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7328b-eb06-4c26-89c6-11ad3eb1481c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:55.956284 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7328b-eb06-4c26-89c6-11ad3eb1481c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:55.963729 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0160a57-ca19-42d0-bbc2-74338feab2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:55.964791 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0160a57-ca19-42d0-bbc2-74338feab2cf admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:55.965264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0160a57-ca19-42d0-bbc2-74338feab2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:55.965723 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0160a57-ca19-42d0-bbc2-74338feab2cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:55.966649 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0160a57-ca19-42d0-bbc2-74338feab2cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:55.970717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0160a57-ca19-42d0-bbc2-74338feab2cf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:56.014129 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1380/2733] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:55 2022] POST /identity/v3/projects => generated 359 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:56.018540 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb2c2b-7f88-452b-961c-90f419ff125c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:56.031404 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb2c2b-7f88-452b-961c-90f419ff125c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:56.106506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738ed35-e009-4221-a909-5bc000fa2940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:56.107477 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738ed35-e009-4221-a909-5bc000fa2940 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:56.108056 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738ed35-e009-4221-a909-5bc000fa2940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:56.108411 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738ed35-e009-4221-a909-5bc000fa2940 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:56.109088 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738ed35-e009-4221-a909-5bc000fa2940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:56.111026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738ed35-e009-4221-a909-5bc000fa2940 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:56.144125 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1354/2734] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:56 2022] POST /identity/v3/users => generated 280 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:56.149108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0160a57-ca19-42d0-bbc2-74338feab2cf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:56.162574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0160a57-ca19-42d0-bbc2-74338feab2cf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:56.169988 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7e9c8-9c4b-4034-94b1-71ad7ad95ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:56.171444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7e9c8-9c4b-4034-94b1-71ad7ad95ede admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:56.171726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7e9c8-9c4b-4034-94b1-71ad7ad95ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:56.172026 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7e9c8-9c4b-4034-94b1-71ad7ad95ede admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:56.173724 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7e9c8-9c4b-4034-94b1-71ad7ad95ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:56.176131 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7e9c8-9c4b-4034-94b1-71ad7ad95ede admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:56.196828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1381/2735] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:56.202690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738ed35-e009-4221-a909-5bc000fa2940 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:56.219134 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738ed35-e009-4221-a909-5bc000fa2940 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:56.226765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37561323-2a7f-4045-8df5-5afb929af117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:56.227940 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37561323-2a7f-4045-8df5-5afb929af117 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:56.228475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37561323-2a7f-4045-8df5-5afb929af117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:56.229017 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37561323-2a7f-4045-8df5-5afb929af117 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:56.230166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37561323-2a7f-4045-8df5-5afb929af117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:56.232232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37561323-2a7f-4045-8df5-5afb929af117 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:56.251809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1355/2736] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:56.257933 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7e9c8-9c4b-4034-94b1-71ad7ad95ede admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:56.278011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7e9c8-9c4b-4034-94b1-71ad7ad95ede admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:56.284420 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a68e64-d4e4-4c75-bcbe-468de68305e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:56.285612 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a68e64-d4e4-4c75-bcbe-468de68305e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:56.286166 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a68e64-d4e4-4c75-bcbe-468de68305e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:56.286606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a68e64-d4e4-4c75-bcbe-468de68305e0 admin admin] PATH_INFO: `/v3/projects/7f624630dc5447f2bcaf8753291533d3/users/c589308abdfc4f9e80cd34c1bc7f3c3d/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:56.314283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a68e64-d4e4-4c75-bcbe-468de68305e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f624630dc5447f2bcaf8753291533d3, user_id=c589308abdfc4f9e80cd34c1bc7f3c3d, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:56.316941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a68e64-d4e4-4c75-bcbe-468de68305e0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:56.334073 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1382/2737] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:56 2022] PUT /identity/v3/projects/7f624630dc5447f2bcaf8753291533d3/users/c589308abdfc4f9e80cd34c1bc7f3c3d/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:56.352103 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e8bed-adda-4d5c-914f-f539661e6bc7 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:56.352643 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e8bed-adda-4d5c-914f-f539661e6bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:56.353061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e8bed-adda-4d5c-914f-f539661e6bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:56.394513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-472e8bed-adda-4d5c-914f-f539661e6bc7 None None] MFA Rules not processed for user `c589308abdfc4f9e80cd34c1bc7f3c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:56.397889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-472e8bed-adda-4d5c-914f-f539661e6bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:56.506813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1356/2738] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:56.532171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a68e64-d4e4-4c75-bcbe-468de68305e0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:56.534039 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f4a68e64-d4e4-4c75-bcbe-468de68305e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:56.552602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e8bed-adda-4d5c-914f-f539661e6bc7 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:56.554570 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-472e8bed-adda-4d5c-914f-f539661e6bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:56.625801 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:56.628375 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a68e64-d4e4-4c75-bcbe-468de68305e0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:56.633668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5337f79-afc9-4d16-8aee-1abff26b5326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:56.634479 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5337f79-afc9-4d16-8aee-1abff26b5326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:56.634760 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5337f79-afc9-4d16-8aee-1abff26b5326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:56.635096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5337f79-afc9-4d16-8aee-1abff26b5326 admin admin] PATH_INFO: `/v3/users/eff99b0038464462a133b175245494cb` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:56.636485 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5337f79-afc9-4d16-8aee-1abff26b5326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eff99b0038464462a133b175245494cb)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:56.638447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5337f79-afc9-4d16-8aee-1abff26b5326 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:56.647281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e8bed-adda-4d5c-914f-f539661e6bc7 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:56.652039 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d14c5b-6862-4d85-b5e0-61274598e515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:56.652672 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d14c5b-6862-4d85-b5e0-61274598e515 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:56.652942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d14c5b-6862-4d85-b5e0-61274598e515 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:56.653239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d14c5b-6862-4d85-b5e0-61274598e515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:56.665793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d14c5b-6862-4d85-b5e0-61274598e515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:56.667534 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d14c5b-6862-4d85-b5e0-61274598e515 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:56.678172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1357/2739] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:56 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:56.707672 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d14c5b-6862-4d85-b5e0-61274598e515 service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:56.720750 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a5337f79-afc9-4d16-8aee-1abff26b5326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eff99b0038464462a133b175245494cb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:56.721151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d14c5b-6862-4d85-b5e0-61274598e515 service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:56.726190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de2eb9-3c13-46ec-97a3-f7aa465533b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:56.726988 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de2eb9-3c13-46ec-97a3-f7aa465533b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:56.727391 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de2eb9-3c13-46ec-97a3-f7aa465533b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:56.727690 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de2eb9-3c13-46ec-97a3-f7aa465533b6 admin admin] PATH_INFO: `/v3/users/c589308abdfc4f9e80cd34c1bc7f3c3d` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:56.729256 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de2eb9-3c13-46ec-97a3-f7aa465533b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c589308abdfc4f9e80cd34c1bc7f3c3d)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:56.731179 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a5337f79-afc9-4d16-8aee-1abff26b5326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eff99b0038464462a133b175245494cb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:56.731470 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de2eb9-3c13-46ec-97a3-f7aa465533b6 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:56.732198 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a5337f79-afc9-4d16-8aee-1abff26b5326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eff99b0038464462a133b175245494cb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:56.743224 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:56.751564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a5337f79-afc9-4d16-8aee-1abff26b5326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eff99b0038464462a133b175245494cb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:56.775139 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a5337f79-afc9-4d16-8aee-1abff26b5326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eff99b0038464462a133b175245494cb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:56.788226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1383/2740] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:56 2022] DELETE /identity/v3/users/eff99b0038464462a133b175245494cb => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:56.793403 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebd6bb-5c44-4347-87f6-fd66ef780398 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:56.793696 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebd6bb-5c44-4347-87f6-fd66ef780398 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:56.793946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebd6bb-5c44-4347-87f6-fd66ef780398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:56.805807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e9de2eb9-3c13-46ec-97a3-f7aa465533b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c589308abdfc4f9e80cd34c1bc7f3c3d'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:56.816377 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f7ebd6bb-5c44-4347-87f6-fd66ef780398 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:56.817665 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f7ebd6bb-5c44-4347-87f6-fd66ef780398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:56.824811 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e9de2eb9-3c13-46ec-97a3-f7aa465533b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c589308abdfc4f9e80cd34c1bc7f3c3d'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:56.833241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e9de2eb9-3c13-46ec-97a3-f7aa465533b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c589308abdfc4f9e80cd34c1bc7f3c3d'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:56.844338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e9de2eb9-3c13-46ec-97a3-f7aa465533b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c589308abdfc4f9e80cd34c1bc7f3c3d'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:56.845506 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-e9de2eb9-3c13-46ec-97a3-f7aa465533b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c589308abdfc4f9e80cd34c1bc7f3c3d'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:56.857003 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1358/2741] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:56 2022] DELETE /identity/v3/users/c589308abdfc4f9e80cd34c1bc7f3c3d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:56.862928 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb2fd1-d7e3-4428-acd2-f784da60b170 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:56.863252 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb2fd1-d7e3-4428-acd2-f784da60b170 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:56.863554 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb2fd1-d7e3-4428-acd2-f784da60b170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:56.885537 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-cbdb2fd1-d7e3-4428-acd2-f784da60b170 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:56.887475 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-cbdb2fd1-d7e3-4428-acd2-f784da60b170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:56.901913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1384/2742] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:56.918331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebd6bb-5c44-4347-87f6-fd66ef780398 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:56.920102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f7ebd6bb-5c44-4347-87f6-fd66ef780398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:56.990612 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebd6bb-5c44-4347-87f6-fd66ef780398 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:56.999080 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1359/2743] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:57.012928 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb2fd1-d7e3-4428-acd2-f784da60b170 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:57.061102 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025240f-9c34-4cfc-a567-880a09c1813a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:57.062092 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025240f-9c34-4cfc-a567-880a09c1813a service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:57.062717 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025240f-9c34-4cfc-a567-880a09c1813a service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:57.063166 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025240f-9c34-4cfc-a567-880a09c1813a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:57.085883 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025240f-9c34-4cfc-a567-880a09c1813a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:57.089556 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025240f-9c34-4cfc-a567-880a09c1813a service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:57.105123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb2fd1-d7e3-4428-acd2-f784da60b170 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:57.107084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1385/2744] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:57.111192 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b5a51-0efd-407b-a927-9340de3cd1a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:57.112189 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b5a51-0efd-407b-a927-9340de3cd1a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:57.112666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b5a51-0efd-407b-a927-9340de3cd1a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:57.113080 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b5a51-0efd-407b-a927-9340de3cd1a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:57.128581 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b5a51-0efd-407b-a927-9340de3cd1a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:57.130097 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b5a51-0efd-407b-a927-9340de3cd1a9 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:57.134615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:57.145347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1360/2745] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:57.295023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025240f-9c34-4cfc-a567-880a09c1813a service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:57.298013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8025240f-9c34-4cfc-a567-880a09c1813a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:57.321730 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025240f-9c34-4cfc-a567-880a09c1813a service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:57.327333 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5dc955-448a-404c-b893-7bdcbfdf22f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:57.328021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5dc955-448a-404c-b893-7bdcbfdf22f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:57.328425 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5dc955-448a-404c-b893-7bdcbfdf22f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:57.328646 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5dc955-448a-404c-b893-7bdcbfdf22f8 admin admin] PATH_INFO: `/v3/users/a20de45f4e9643f1bde1be7389a09b00` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:57.329896 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5dc955-448a-404c-b893-7bdcbfdf22f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a20de45f4e9643f1bde1be7389a09b00)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:57.331935 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5dc955-448a-404c-b893-7bdcbfdf22f8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:57.401245 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:57.419249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cd5dc955-448a-404c-b893-7bdcbfdf22f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a20de45f4e9643f1bde1be7389a09b00'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.432543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cd5dc955-448a-404c-b893-7bdcbfdf22f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a20de45f4e9643f1bde1be7389a09b00'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.434652 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cd5dc955-448a-404c-b893-7bdcbfdf22f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a20de45f4e9643f1bde1be7389a09b00'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.453877 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cd5dc955-448a-404c-b893-7bdcbfdf22f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a20de45f4e9643f1bde1be7389a09b00'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.464997 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cd5dc955-448a-404c-b893-7bdcbfdf22f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a20de45f4e9643f1bde1be7389a09b00'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.479468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1386/2746] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:57 2022] DELETE /identity/v3/users/a20de45f4e9643f1bde1be7389a09b00 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:57.489397 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51eab68-30f0-426a-8500-c26690f17152 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:57.489986 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51eab68-30f0-426a-8500-c26690f17152 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:57.490611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51eab68-30f0-426a-8500-c26690f17152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:57.519572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-b51eab68-30f0-426a-8500-c26690f17152 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:57.521179 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b51eab68-30f0-426a-8500-c26690f17152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:57.674260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5dc955-448a-404c-b893-7bdcbfdf22f8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:57.677458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cd5dc955-448a-404c-b893-7bdcbfdf22f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:57.821717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1361/2747] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:57.827833 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51eab68-30f0-426a-8500-c26690f17152 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:57.830008 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b51eab68-30f0-426a-8500-c26690f17152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:57.850085 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5dc955-448a-404c-b893-7bdcbfdf22f8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:57.850804 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51eab68-30f0-426a-8500-c26690f17152 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:57.856908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:57.857183 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:57.857732 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:57.858002 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:57.858174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:57.858354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:57.858684 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] PATH_INFO: `/v3/projects/7f624630dc5447f2bcaf8753291533d3` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:57.858965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] PATH_INFO: `/v3/projects/dad91d3c81754c309b0f685fd388e599` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:57.860346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f624630dc5447f2bcaf8753291533d3)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:57.860525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dad91d3c81754c309b0f685fd388e599)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:57.862801 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:57.862993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:57.926167 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad91d3c81754c309b0f685fd388e599', 'request_id': 'req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.928274 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f624630dc5447f2bcaf8753291533d3', 'request_id': 'req-6f9fcce1-8e12-4e76-b128-b035b07c1b84'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.934641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad91d3c81754c309b0f685fd388e599', 'request_id': 'req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.944379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad91d3c81754c309b0f685fd388e599', 'request_id': 'req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.949697 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f624630dc5447f2bcaf8753291533d3', 'request_id': 'req-6f9fcce1-8e12-4e76-b128-b035b07c1b84'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.961547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] The token cache is being invalidate because project dad91d3c81754c309b0f685fd388e599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:57.962277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.962461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f624630dc5447f2bcaf8753291533d3', 'request_id': 'req-6f9fcce1-8e12-4e76-b128-b035b07c1b84'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.963331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.965322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1387/2748] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:57 2022] DELETE /identity/v3/projects/dad91d3c81754c309b0f685fd388e599 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:57.966636 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:57.968373 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:57.986771 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] The token cache is being invalidate because project 7f624630dc5447f2bcaf8753291533d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:57.987099 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.988050 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:57.989879 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1362/2749] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:57 2022] DELETE /identity/v3/projects/7f624630dc5447f2bcaf8753291533d3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:57.990793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:57.992578 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:58.045714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:58.064505 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5675b88-4d1a-4614-9c24-b5f8cb9e7f06 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:58.071273 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8a2fb-31e9-4a59-9d1d-ccab0494758d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:58.072105 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8a2fb-31e9-4a59-9d1d-ccab0494758d service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:58.072567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8a2fb-31e9-4a59-9d1d-ccab0494758d service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:58.073049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8a2fb-31e9-4a59-9d1d-ccab0494758d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:58.075679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-39b8a2fb-31e9-4a59-9d1d-ccab0494758d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:58.108895 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fcce1-8e12-4e76-b128-b035b07c1b84 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:58.141416 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:58.164462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8a2fb-31e9-4a59-9d1d-ccab0494758d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:58.165922 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8a2fb-31e9-4a59-9d1d-ccab0494758d service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:58.176814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1388/2750] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:58.178047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e74c1-662b-45e6-b319-f53a59ce56e7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:58.178330 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e74c1-662b-45e6-b319-f53a59ce56e7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:58.178605 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e74c1-662b-45e6-b319-f53a59ce56e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:58.189485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9241635-5b0a-4e1e-b05b-2c54fbcfd095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:58.190350 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9241635-5b0a-4e1e-b05b-2c54fbcfd095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:58.190931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9241635-5b0a-4e1e-b05b-2c54fbcfd095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:58.191671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9241635-5b0a-4e1e-b05b-2c54fbcfd095 admin admin] PATH_INFO: `/v3/users/24f3ccc2a2d642e89fe346316988a238` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:58.193520 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9241635-5b0a-4e1e-b05b-2c54fbcfd095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24f3ccc2a2d642e89fe346316988a238)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:58.197884 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9241635-5b0a-4e1e-b05b-2c54fbcfd095 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:58.204764 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-da1e74c1-662b-45e6-b319-f53a59ce56e7 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:58.206096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-da1e74c1-662b-45e6-b319-f53a59ce56e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:58.215714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1389/2751] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:58.223626 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324a946-f403-429f-a894-d432c731d419 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:58.223957 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324a946-f403-429f-a894-d432c731d419 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:58.224200 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324a946-f403-429f-a894-d432c731d419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:58.261904 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3324a946-f403-429f-a894-d432c731d419 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:58.264069 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3324a946-f403-429f-a894-d432c731d419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:58.282376 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1390/2752] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:58.290323 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324a946-f403-429f-a894-d432c731d419 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:58.303956 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324a946-f403-429f-a894-d432c731d419 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:58.306990 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d9241635-5b0a-4e1e-b05b-2c54fbcfd095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24f3ccc2a2d642e89fe346316988a238'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:58.310821 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51117c-9a49-4d2b-871a-204fea9f25cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:58.311627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51117c-9a49-4d2b-871a-204fea9f25cd admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:58.311965 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51117c-9a49-4d2b-871a-204fea9f25cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:58.312251 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51117c-9a49-4d2b-871a-204fea9f25cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:58.313027 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51117c-9a49-4d2b-871a-204fea9f25cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:58.313331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51117c-9a49-4d2b-871a-204fea9f25cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:58.316358 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51117c-9a49-4d2b-871a-204fea9f25cd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:58.333585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d9241635-5b0a-4e1e-b05b-2c54fbcfd095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24f3ccc2a2d642e89fe346316988a238'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:58.339038 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1391/2753] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:58.344369 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51117c-9a49-4d2b-871a-204fea9f25cd admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:58.350153 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d9241635-5b0a-4e1e-b05b-2c54fbcfd095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24f3ccc2a2d642e89fe346316988a238'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:58.357014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51117c-9a49-4d2b-871a-204fea9f25cd admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:58.361835 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d9241635-5b0a-4e1e-b05b-2c54fbcfd095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24f3ccc2a2d642e89fe346316988a238'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:58.362745 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-d9241635-5b0a-4e1e-b05b-2c54fbcfd095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24f3ccc2a2d642e89fe346316988a238'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:58.364499 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82092e-76a0-4dab-8d51-18c1dd900281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:58.365684 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82092e-76a0-4dab-8d51-18c1dd900281 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:58.366664 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82092e-76a0-4dab-8d51-18c1dd900281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:58.366950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82092e-76a0-4dab-8d51-18c1dd900281 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:58.368101 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82092e-76a0-4dab-8d51-18c1dd900281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:58.370641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82092e-76a0-4dab-8d51-18c1dd900281 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:58.371593 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1363/2754] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:57 2022] DELETE /identity/v3/users/24f3ccc2a2d642e89fe346316988a238 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:58.426997 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1392/2755] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:58 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:58.435047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9241635-5b0a-4e1e-b05b-2c54fbcfd095 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:58.437049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-d9241635-5b0a-4e1e-b05b-2c54fbcfd095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:58.543062 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9241635-5b0a-4e1e-b05b-2c54fbcfd095 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:58.611041 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82092e-76a0-4dab-8d51-18c1dd900281 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:58.616452 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8f82092e-76a0-4dab-8d51-18c1dd900281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:58.621298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:58.637275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b943f-1350-43fe-a89c-628910486edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:58.637981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b943f-1350-43fe-a89c-628910486edb admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:58.638568 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b943f-1350-43fe-a89c-628910486edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:58.638939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b943f-1350-43fe-a89c-628910486edb admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:58.639877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b943f-1350-43fe-a89c-628910486edb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:58.642972 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b943f-1350-43fe-a89c-628910486edb admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:58.675634 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1364/2756] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:58 2022] POST /identity/v3/users => generated 287 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:58.682390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b943f-1350-43fe-a89c-628910486edb admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:58.702790 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b943f-1350-43fe-a89c-628910486edb admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:58.713678 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db2b56-8a07-4022-978e-e6fb62bc33c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:58.714700 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db2b56-8a07-4022-978e-e6fb62bc33c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:58.715226 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db2b56-8a07-4022-978e-e6fb62bc33c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:58.715706 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db2b56-8a07-4022-978e-e6fb62bc33c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:58.717596 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db2b56-8a07-4022-978e-e6fb62bc33c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:58.721813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db2b56-8a07-4022-978e-e6fb62bc33c3 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:58.740347 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1365/2757] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:58.746484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db2b56-8a07-4022-978e-e6fb62bc33c3 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:58.765812 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db2b56-8a07-4022-978e-e6fb62bc33c3 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:58.778517 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56fcae-b78e-49b7-a1b5-b7d105b70801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:58.779148 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56fcae-b78e-49b7-a1b5-b7d105b70801 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:58.779420 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56fcae-b78e-49b7-a1b5-b7d105b70801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:58.779873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56fcae-b78e-49b7-a1b5-b7d105b70801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:58.780545 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56fcae-b78e-49b7-a1b5-b7d105b70801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:58.782843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82092e-76a0-4dab-8d51-18c1dd900281 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:58.783325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56fcae-b78e-49b7-a1b5-b7d105b70801 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:58.791223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5012d9-6215-48b3-b74b-1977f110bfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:58.792374 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5012d9-6215-48b3-b74b-1977f110bfdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:58.793414 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5012d9-6215-48b3-b74b-1977f110bfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:58.793899 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5012d9-6215-48b3-b74b-1977f110bfdb admin admin] PATH_INFO: `/v3/projects/c0ee0111943f427a8d7745468c651166` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:58.796000 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5012d9-6215-48b3-b74b-1977f110bfdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ee0111943f427a8d7745468c651166)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:58.799327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5012d9-6215-48b3-b74b-1977f110bfdb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:58.801348 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1366/2758] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:43:58.807644 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56fcae-b78e-49b7-a1b5-b7d105b70801 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:58.822151 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56fcae-b78e-49b7-a1b5-b7d105b70801 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:58.827539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d5497-2eb6-4df2-9f6e-dc7beed36f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:58.828262 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d5497-2eb6-4df2-9f6e-dc7beed36f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:58.828572 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d5497-2eb6-4df2-9f6e-dc7beed36f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:58.828802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d5497-2eb6-4df2-9f6e-dc7beed36f13 admin admin] PATH_INFO: `/v3/projects/6add1594e65e475d8d6e4b6609c5046f/users/228f30eb00e8460caddc7b34a6138880/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:58.850372 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d5497-2eb6-4df2-9f6e-dc7beed36f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6add1594e65e475d8d6e4b6609c5046f, user_id=228f30eb00e8460caddc7b34a6138880, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:58.854275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d5497-2eb6-4df2-9f6e-dc7beed36f13 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:58.871281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1367/2759] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:58 2022] PUT /identity/v3/projects/6add1594e65e475d8d6e4b6609c5046f/users/228f30eb00e8460caddc7b34a6138880/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:58.872539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d5497-2eb6-4df2-9f6e-dc7beed36f13 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:58.886548 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:58.887683 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-dd7d5497-2eb6-4df2-9f6e-dc7beed36f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:58.888964 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a5012d9-6215-48b3-b74b-1977f110bfdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ee0111943f427a8d7745468c651166', 'request_id': 'req-0a5012d9-6215-48b3-b74b-1977f110bfdb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:58.899093 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a5012d9-6215-48b3-b74b-1977f110bfdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ee0111943f427a8d7745468c651166', 'request_id': 'req-0a5012d9-6215-48b3-b74b-1977f110bfdb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:58.907611 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a5012d9-6215-48b3-b74b-1977f110bfdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ee0111943f427a8d7745468c651166', 'request_id': 'req-0a5012d9-6215-48b3-b74b-1977f110bfdb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:58.926957 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a5012d9-6215-48b3-b74b-1977f110bfdb admin admin] The token cache is being invalidate because project c0ee0111943f427a8d7745468c651166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:58.928150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a5012d9-6215-48b3-b74b-1977f110bfdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:58.928855 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a5012d9-6215-48b3-b74b-1977f110bfdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:58.930962 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1393/2760] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:58 2022] DELETE /identity/v3/projects/c0ee0111943f427a8d7745468c651166 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:58.932216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54afde-24f4-48ac-b4d8-da353d094d54 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:58.932497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54afde-24f4-48ac-b4d8-da353d094d54 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:58.932767 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54afde-24f4-48ac-b4d8-da353d094d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:58.980505 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0e54afde-24f4-48ac-b4d8-da353d094d54 None None] MFA Rules not processed for user `228f30eb00e8460caddc7b34a6138880`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:58.982498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0e54afde-24f4-48ac-b4d8-da353d094d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:58.997298 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d5497-2eb6-4df2-9f6e-dc7beed36f13 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.003120 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.003911 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.004346 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.004711 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] PATH_INFO: `/v3/projects/33f8517a4cc04bc085185a35bb67eab2` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.006022 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33f8517a4cc04bc085185a35bb67eab2)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.009299 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.071811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1394/2761] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:59.073303 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7bb43-ecf5-4492-9c53-2240a68fc27c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.073583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7bb43-ecf5-4492-9c53-2240a68fc27c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.073957 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7bb43-ecf5-4492-9c53-2240a68fc27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.087484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33f8517a4cc04bc085185a35bb67eab2', 'request_id': 'req-ea783d1f-7c78-49fb-8634-f76a39cf79b9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:59.097761 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33f8517a4cc04bc085185a35bb67eab2', 'request_id': 'req-ea783d1f-7c78-49fb-8634-f76a39cf79b9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:59.098482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-39d7bb43-ecf5-4492-9c53-2240a68fc27c None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:59.099721 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-39d7bb43-ecf5-4492-9c53-2240a68fc27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:59.104995 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33f8517a4cc04bc085185a35bb67eab2', 'request_id': 'req-ea783d1f-7c78-49fb-8634-f76a39cf79b9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:59.120664 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] The token cache is being invalidate because project 33f8517a4cc04bc085185a35bb67eab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:59.121863 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:59.122508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:59.124135 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1368/2762] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:58 2022] DELETE /identity/v3/projects/33f8517a4cc04bc085185a35bb67eab2 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:59.124916 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.126735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:59.172679 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:59.178706 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1395/2763] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:59.180734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3607be-8c4c-488a-9017-8af414d4d543 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.181170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3607be-8c4c-488a-9017-8af414d4d543 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.181578 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3607be-8c4c-488a-9017-8af414d4d543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.209157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-de3607be-8c4c-488a-9017-8af414d4d543 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:59.211223 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-de3607be-8c4c-488a-9017-8af414d4d543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:59.224503 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1396/2764] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:59.225348 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea783d1f-7c78-49fb-8634-f76a39cf79b9 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.225523 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316db1b-2592-432e-b3a2-eecb7e2bb0b1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.225699 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316db1b-2592-432e-b3a2-eecb7e2bb0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.226003 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316db1b-2592-432e-b3a2-eecb7e2bb0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.235798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228ce6a-1224-4c04-9aa9-cd2b928fb71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.236555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228ce6a-1224-4c04-9aa9-cd2b928fb71b admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.236922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228ce6a-1224-4c04-9aa9-cd2b928fb71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.237354 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228ce6a-1224-4c04-9aa9-cd2b928fb71b admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.238278 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228ce6a-1224-4c04-9aa9-cd2b928fb71b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.240488 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228ce6a-1224-4c04-9aa9-cd2b928fb71b admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.259527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4316db1b-2592-432e-b3a2-eecb7e2bb0b1 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:59.260802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4316db1b-2592-432e-b3a2-eecb7e2bb0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:59.269218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1397/2765] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:59.270475 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981d696-0b6d-40df-bc51-27df45e52e27 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.270748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981d696-0b6d-40df-bc51-27df45e52e27 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.271103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981d696-0b6d-40df-bc51-27df45e52e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.277121 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1369/2766] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:59 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:59.277121 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228ce6a-1224-4c04-9aa9-cd2b928fb71b admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.297106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d981d696-0b6d-40df-bc51-27df45e52e27 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:59.298341 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d981d696-0b6d-40df-bc51-27df45e52e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:59.350842 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228ce6a-1224-4c04-9aa9-cd2b928fb71b admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.357057 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461277a-7c55-433c-b248-8bf351aad1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.358032 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461277a-7c55-433c-b248-8bf351aad1bf admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.358462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461277a-7c55-433c-b248-8bf351aad1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.358888 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461277a-7c55-433c-b248-8bf351aad1bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.360334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461277a-7c55-433c-b248-8bf351aad1bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:59.360858 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461277a-7c55-433c-b248-8bf351aad1bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.361283 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:59.364333 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461277a-7c55-433c-b248-8bf351aad1bf admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.378552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1398/2767] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:59.386675 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1370/2768] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:59.386952 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981d696-0b6d-40df-bc51-27df45e52e27 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.387130 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461277a-7c55-433c-b248-8bf351aad1bf admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.396413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981d696-0b6d-40df-bc51-27df45e52e27 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.401727 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a54a30-98cf-417a-8c54-85ad9bf55a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.402348 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a54a30-98cf-417a-8c54-85ad9bf55a1e admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.402625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a54a30-98cf-417a-8c54-85ad9bf55a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.402888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a54a30-98cf-417a-8c54-85ad9bf55a1e admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.403655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a54a30-98cf-417a-8c54-85ad9bf55a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.405895 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a54a30-98cf-417a-8c54-85ad9bf55a1e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.407489 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461277a-7c55-433c-b248-8bf351aad1bf admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.416989 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2e00b-9568-4148-b24e-9c2f83f8d99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.418079 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2e00b-9568-4148-b24e-9c2f83f8d99c admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.418458 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2e00b-9568-4148-b24e-9c2f83f8d99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.419285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2e00b-9568-4148-b24e-9c2f83f8d99c admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.422805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2e00b-9568-4148-b24e-9c2f83f8d99c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:59.423584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2e00b-9568-4148-b24e-9c2f83f8d99c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.427797 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2e00b-9568-4148-b24e-9c2f83f8d99c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.442095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1399/2769] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:59 2022] POST /identity/v3/users => generated 294 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:59.442095 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a54a30-98cf-417a-8c54-85ad9bf55a1e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.453185 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1371/2770] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:59.453185 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2e00b-9568-4148-b24e-9c2f83f8d99c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.458356 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a54a30-98cf-417a-8c54-85ad9bf55a1e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.466326 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f2dc0-dbf9-4fdf-b4d0-1b0e4481d725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.467564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f2dc0-dbf9-4fdf-b4d0-1b0e4481d725 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.468312 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f2dc0-dbf9-4fdf-b4d0-1b0e4481d725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.468768 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f2dc0-dbf9-4fdf-b4d0-1b0e4481d725 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.469605 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2e00b-9568-4148-b24e-9c2f83f8d99c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.470085 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f2dc0-dbf9-4fdf-b4d0-1b0e4481d725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.475266 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f2dc0-dbf9-4fdf-b4d0-1b0e4481d725 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.475728 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc98d0a-fa6b-467c-bcd0-784093394df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.476433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc98d0a-fa6b-467c-bcd0-784093394df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.476713 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc98d0a-fa6b-467c-bcd0-784093394df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.476981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc98d0a-fa6b-467c-bcd0-784093394df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.477663 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc98d0a-fa6b-467c-bcd0-784093394df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.479244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc98d0a-fa6b-467c-bcd0-784093394df3 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.497465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1372/2771] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:59.497465 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc98d0a-fa6b-467c-bcd0-784093394df3 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.505459 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc98d0a-fa6b-467c-bcd0-784093394df3 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.511611 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77fc49-fb38-484a-886c-d65f49aad7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.512284 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77fc49-fb38-484a-886c-d65f49aad7d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.512577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77fc49-fb38-484a-886c-d65f49aad7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.512843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77fc49-fb38-484a-886c-d65f49aad7d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.513484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77fc49-fb38-484a-886c-d65f49aad7d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.515385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae77fc49-fb38-484a-886c-d65f49aad7d6 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.517212 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1400/2772] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:59 2022] POST /identity/v3/projects => generated 363 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:59.518046 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f2dc0-dbf9-4fdf-b4d0-1b0e4481d725 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.551971 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1373/2773] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:59 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:59.553250 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77fc49-fb38-484a-886c-d65f49aad7d6 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.603595 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f2dc0-dbf9-4fdf-b4d0-1b0e4481d725 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.641856 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77fc49-fb38-484a-886c-d65f49aad7d6 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.648115 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2c03e-4674-4448-b24f-9f3d4166d94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.649014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2c03e-4674-4448-b24f-9f3d4166d94e admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.649476 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2c03e-4674-4448-b24f-9f3d4166d94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.649888 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2c03e-4674-4448-b24f-9f3d4166d94e admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.650665 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2c03e-4674-4448-b24f-9f3d4166d94e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.652873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2c03e-4674-4448-b24f-9f3d4166d94e admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.664870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:59.699595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1374/2774] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:59 2022] POST /identity/v3/users => generated 282 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:59.699595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2c03e-4674-4448-b24f-9f3d4166d94e admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.710232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8769663-85e1-448e-b7f8-cbeaf6c31cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.710902 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2c03e-4674-4448-b24f-9f3d4166d94e admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.711765 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8769663-85e1-448e-b7f8-cbeaf6c31cf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.712481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8769663-85e1-448e-b7f8-cbeaf6c31cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.712748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8769663-85e1-448e-b7f8-cbeaf6c31cf6 admin admin] PATH_INFO: `/v3/projects/5a883dfcf6744f07ac3759a9568baf98/users/7c7477061e6f47a8a71e0d38459dd472/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.716825 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263fe2ba-a307-4fa4-8126-a904347f04ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.719963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fe2ba-a307-4fa4-8126-a904347f04ea admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.719963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fe2ba-a307-4fa4-8126-a904347f04ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.719963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fe2ba-a307-4fa4-8126-a904347f04ea admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.719963 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263fe2ba-a307-4fa4-8126-a904347f04ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.722237 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263fe2ba-a307-4fa4-8126-a904347f04ea admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.741337 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8769663-85e1-448e-b7f8-cbeaf6c31cf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a883dfcf6744f07ac3759a9568baf98, user_id=7c7477061e6f47a8a71e0d38459dd472, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.746677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8769663-85e1-448e-b7f8-cbeaf6c31cf6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.749798 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1375/2775] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:59 2022] POST /identity/v3/users => generated 293 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:59.751493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263fe2ba-a307-4fa4-8126-a904347f04ea admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.769056 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263fe2ba-a307-4fa4-8126-a904347f04ea admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.776925 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1401/2776] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:59 2022] PUT /identity/v3/projects/5a883dfcf6744f07ac3759a9568baf98/users/7c7477061e6f47a8a71e0d38459dd472/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:59.781079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8769663-85e1-448e-b7f8-cbeaf6c31cf6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.837764 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cb333-b4bc-42fb-841f-36f704349605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.838560 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cb333-b4bc-42fb-841f-36f704349605 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.839064 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cb333-b4bc-42fb-841f-36f704349605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.839388 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cb333-b4bc-42fb-841f-36f704349605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.840080 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cb333-b4bc-42fb-841f-36f704349605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.841708 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cb333-b4bc-42fb-841f-36f704349605 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.852226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:59.853167 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1376/2777] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:59.854037 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cb333-b4bc-42fb-841f-36f704349605 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.867582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cb333-b4bc-42fb-841f-36f704349605 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.875669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce8cd4-e5b9-4e0c-a684-c42226f362be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.876695 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce8cd4-e5b9-4e0c-a684-c42226f362be admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.877368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce8cd4-e5b9-4e0c-a684-c42226f362be admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.878080 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce8cd4-e5b9-4e0c-a684-c42226f362be admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.878923 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce8cd4-e5b9-4e0c-a684-c42226f362be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.880621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce8cd4-e5b9-4e0c-a684-c42226f362be admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.881926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8769663-85e1-448e-b7f8-cbeaf6c31cf6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.893859 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6020dc6-3dd4-4d75-bd24-793c6b30e723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.894379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1377/2778] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:59.895434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce8cd4-e5b9-4e0c-a684-c42226f362be admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.898805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6020dc6-3dd4-4d75-bd24-793c6b30e723 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.899548 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6020dc6-3dd4-4d75-bd24-793c6b30e723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.901561 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6020dc6-3dd4-4d75-bd24-793c6b30e723 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.902239 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6020dc6-3dd4-4d75-bd24-793c6b30e723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.903875 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6020dc6-3dd4-4d75-bd24-793c6b30e723 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.906962 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce8cd4-e5b9-4e0c-a684-c42226f362be admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.912795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62c2fe-caf3-482f-8b7d-f5ba7a20204f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.913455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62c2fe-caf3-482f-8b7d-f5ba7a20204f admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.913833 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62c2fe-caf3-482f-8b7d-f5ba7a20204f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.914127 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62c2fe-caf3-482f-8b7d-f5ba7a20204f admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.914809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62c2fe-caf3-482f-8b7d-f5ba7a20204f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.916497 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62c2fe-caf3-482f-8b7d-f5ba7a20204f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.922307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1402/2779] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:59.922307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6020dc6-3dd4-4d75-bd24-793c6b30e723 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.931413 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1378/2780] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:59.932448 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62c2fe-caf3-482f-8b7d-f5ba7a20204f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.937830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6020dc6-3dd4-4d75-bd24-793c6b30e723 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.944152 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2ccc2-376d-4d04-a957-097cbed64436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.945134 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2ccc2-376d-4d04-a957-097cbed64436 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.945773 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2ccc2-376d-4d04-a957-097cbed64436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.946293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2ccc2-376d-4d04-a957-097cbed64436 admin admin] PATH_INFO: `/v3/projects/5a883dfcf6744f07ac3759a9568baf98/users/7c7477061e6f47a8a71e0d38459dd472/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.947526 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62c2fe-caf3-482f-8b7d-f5ba7a20204f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:59.948891 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2ccc2-376d-4d04-a957-097cbed64436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a883dfcf6744f07ac3759a9568baf98, user_id=7c7477061e6f47a8a71e0d38459dd472, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.951736 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2ccc2-376d-4d04-a957-097cbed64436 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.955461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d14a69-86df-49bd-80ee-09e33a21fd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:59.956377 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d14a69-86df-49bd-80ee-09e33a21fd36 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.956850 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d14a69-86df-49bd-80ee-09e33a21fd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.957330 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d14a69-86df-49bd-80ee-09e33a21fd36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:59.958419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d14a69-86df-49bd-80ee-09e33a21fd36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:59.961190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d14a69-86df-49bd-80ee-09e33a21fd36 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:59.963927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1403/2781] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:59 2022] PUT /identity/v3/projects/5a883dfcf6744f07ac3759a9568baf98/users/7c7477061e6f47a8a71e0d38459dd472/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:59.968986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2ccc2-376d-4d04-a957-097cbed64436 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:59.986738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1379/2782] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:59.986738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c386-6f36-435d-9519-a490a385e738 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:59.986738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c386-6f36-435d-9519-a490a385e738 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:59.986738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c386-6f36-435d-9519-a490a385e738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:00.026047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-0823c386-6f36-435d-9519-a490a385e738 None None] MFA Rules not processed for user `7c7477061e6f47a8a71e0d38459dd472`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:00.027707 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0823c386-6f36-435d-9519-a490a385e738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:00.046352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2ccc2-376d-4d04-a957-097cbed64436 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:00.053983 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c5a32-9f5d-438d-bfb8-c7e3f0fd2f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:00.055294 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c5a32-9f5d-438d-bfb8-c7e3f0fd2f07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:00.055760 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c5a32-9f5d-438d-bfb8-c7e3f0fd2f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:00.056174 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c5a32-9f5d-438d-bfb8-c7e3f0fd2f07 admin admin] PATH_INFO: `/v3/projects/782af202d5ea43838897b844a66093f6/users/e1804a4ef1974e0d8d7d062bd381efc6/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:00.079692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c5a32-9f5d-438d-bfb8-c7e3f0fd2f07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=782af202d5ea43838897b844a66093f6, user_id=e1804a4ef1974e0d8d7d062bd381efc6, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:00.084842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c5a32-9f5d-438d-bfb8-c7e3f0fd2f07 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:00.101362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1404/2783] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:59 2022] PUT /identity/v3/projects/782af202d5ea43838897b844a66093f6/users/e1804a4ef1974e0d8d7d062bd381efc6/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:00.109299 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c5a32-9f5d-438d-bfb8-c7e3f0fd2f07 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:00.132175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1380/2784] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:43:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:00.133868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83866e2-c357-4728-a4bb-f536eba35828 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:00.134123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83866e2-c357-4728-a4bb-f536eba35828 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:00.134394 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83866e2-c357-4728-a4bb-f536eba35828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:00.174717 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-f83866e2-c357-4728-a4bb-f536eba35828 None None] MFA Rules not processed for user `e1804a4ef1974e0d8d7d062bd381efc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:00.176362 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-f83866e2-c357-4728-a4bb-f536eba35828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:00.189438 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:00.198919 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c5a32-9f5d-438d-bfb8-c7e3f0fd2f07 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:00.206447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778949e-af44-4b02-88a7-389662e7461e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:00.207071 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778949e-af44-4b02-88a7-389662e7461e admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:00.207342 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778949e-af44-4b02-88a7-389662e7461e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:00.207611 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778949e-af44-4b02-88a7-389662e7461e admin admin] PATH_INFO: `/v3/projects/0410ff3e18434c5e8a2e7b33587c487c/users/ebf5138b4bf84239a69919f3b49ef99e/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:00.232718 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778949e-af44-4b02-88a7-389662e7461e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0410ff3e18434c5e8a2e7b33587c487c, user_id=ebf5138b4bf84239a69919f3b49ef99e, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:00.235447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778949e-af44-4b02-88a7-389662e7461e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:00.254583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1405/2785] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:00 2022] PUT /identity/v3/projects/0410ff3e18434c5e8a2e7b33587c487c/users/ebf5138b4bf84239a69919f3b49ef99e/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:00.256096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778949e-af44-4b02-88a7-389662e7461e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:00.259170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9778949e-af44-4b02-88a7-389662e7461e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:00.271743 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1381/2786] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:00.274621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3193938-8241-4430-a120-f578f0f7b9d1 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:00.274969 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3193938-8241-4430-a120-f578f0f7b9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:00.275362 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3193938-8241-4430-a120-f578f0f7b9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:00.314017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:00.322301 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-a3193938-8241-4430-a120-f578f0f7b9d1 None None] MFA Rules not processed for user `ebf5138b4bf84239a69919f3b49ef99e`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:00.324325 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a3193938-8241-4430-a120-f578f0f7b9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:00.346529 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778949e-af44-4b02-88a7-389662e7461e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:00.354025 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6f878-6ca8-47cb-8330-a7875245690b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:00.355387 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6f878-6ca8-47cb-8330-a7875245690b service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:00.356089 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6f878-6ca8-47cb-8330-a7875245690b service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:00.356684 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6f878-6ca8-47cb-8330-a7875245690b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:00.360427 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-58d6f878-6ca8-47cb-8330-a7875245690b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:00.408592 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1382/2787] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:00.410024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3193938-8241-4430-a120-f578f0f7b9d1 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:00.411735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a3193938-8241-4430-a120-f578f0f7b9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:00.432746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3193938-8241-4430-a120-f578f0f7b9d1 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:00.438816 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb95ab-1f32-4ad3-a9e8-20a4039744c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:00.439796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb95ab-1f32-4ad3-a9e8-20a4039744c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:00.440276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb95ab-1f32-4ad3-a9e8-20a4039744c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:00.440805 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb95ab-1f32-4ad3-a9e8-20a4039744c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:00.441769 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6f878-6ca8-47cb-8330-a7875245690b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:00.443226 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6f878-6ca8-47cb-8330-a7875245690b service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:00.457952 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1406/2788] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:00.520951 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb95ab-1f32-4ad3-a9e8-20a4039744c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:00.522456 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb95ab-1f32-4ad3-a9e8-20a4039744c3 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:00.532568 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1383/2789] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:00.795542 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6f878-6ca8-47cb-8330-a7875245690b service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:00.809096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6f878-6ca8-47cb-8330-a7875245690b service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:00.819777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff24a6a-9cf4-42a5-b11c-0099bce865d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:00.821644 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff24a6a-9cf4-42a5-b11c-0099bce865d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:00.822098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff24a6a-9cf4-42a5-b11c-0099bce865d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:00.822581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff24a6a-9cf4-42a5-b11c-0099bce865d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:00.840559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff24a6a-9cf4-42a5-b11c-0099bce865d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:00.843242 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff24a6a-9cf4-42a5-b11c-0099bce865d3 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:00.860597 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1407/2790] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:03.283270 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb95ab-1f32-4ad3-a9e8-20a4039744c3 service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:03.329413 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:03.351648 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb95ab-1f32-4ad3-a9e8-20a4039744c3 service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:03.356483 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd7ab0-527b-4d77-95c4-d774ed4ea9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:03.357117 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd7ab0-527b-4d77-95c4-d774ed4ea9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:03.357396 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd7ab0-527b-4d77-95c4-d774ed4ea9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:03.357660 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd7ab0-527b-4d77-95c4-d774ed4ea9dd admin admin] PATH_INFO: `/v3/users/228f30eb00e8460caddc7b34a6138880` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:03.358769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd7ab0-527b-4d77-95c4-d774ed4ea9dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=228f30eb00e8460caddc7b34a6138880)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:03.360738 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd7ab0-527b-4d77-95c4-d774ed4ea9dd admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:03.417214 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a3cd7ab0-527b-4d77-95c4-d774ed4ea9dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '228f30eb00e8460caddc7b34a6138880'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:03.431161 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a3cd7ab0-527b-4d77-95c4-d774ed4ea9dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '228f30eb00e8460caddc7b34a6138880'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:03.438372 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a3cd7ab0-527b-4d77-95c4-d774ed4ea9dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '228f30eb00e8460caddc7b34a6138880'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:03.447585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a3cd7ab0-527b-4d77-95c4-d774ed4ea9dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '228f30eb00e8460caddc7b34a6138880'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:03.448270 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-a3cd7ab0-527b-4d77-95c4-d774ed4ea9dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '228f30eb00e8460caddc7b34a6138880'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:03.457074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1384/2791] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:03 2022] DELETE /identity/v3/users/228f30eb00e8460caddc7b34a6138880 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:03.462799 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5419a0-1bd1-43f1-8710-825c61686ca0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:03.463379 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5419a0-1bd1-43f1-8710-825c61686ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:03.463485 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5419a0-1bd1-43f1-8710-825c61686ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:03.483131 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cc5419a0-1bd1-43f1-8710-825c61686ca0 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:03.484394 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cc5419a0-1bd1-43f1-8710-825c61686ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:03.553409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1408/2792] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:03.564603 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd7ab0-527b-4d77-95c4-d774ed4ea9dd admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:03.566124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a3cd7ab0-527b-4d77-95c4-d774ed4ea9dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:03.656701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd7ab0-527b-4d77-95c4-d774ed4ea9dd admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:03.661956 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e5887-998d-4fc7-8b69-54c407b6f481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:03.662826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e5887-998d-4fc7-8b69-54c407b6f481 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:03.663414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e5887-998d-4fc7-8b69-54c407b6f481 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:03.663780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e5887-998d-4fc7-8b69-54c407b6f481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:03.675534 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e5887-998d-4fc7-8b69-54c407b6f481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:03.677012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e5887-998d-4fc7-8b69-54c407b6f481 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:03.687621 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1385/2793] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:04.025082 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5419a0-1bd1-43f1-8710-825c61686ca0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:04.026651 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cc5419a0-1bd1-43f1-8710-825c61686ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:04.043342 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5419a0-1bd1-43f1-8710-825c61686ca0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:04.049481 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:04.050136 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:04.050412 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:04.050679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] PATH_INFO: `/v3/projects/6add1594e65e475d8d6e4b6609c5046f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:04.051840 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6add1594e65e475d8d6e4b6609c5046f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:04.053768 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:04.118211 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6add1594e65e475d8d6e4b6609c5046f', 'request_id': 'req-3dbdd62e-fc75-4bbc-989b-8090246f8513'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.124411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6add1594e65e475d8d6e4b6609c5046f', 'request_id': 'req-3dbdd62e-fc75-4bbc-989b-8090246f8513'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.131474 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6add1594e65e475d8d6e4b6609c5046f', 'request_id': 'req-3dbdd62e-fc75-4bbc-989b-8090246f8513'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.143024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:04.149448 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] The token cache is being invalidate because project 6add1594e65e475d8d6e4b6609c5046f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:04.149793 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.151116 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.153079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1409/2794] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:04 2022] DELETE /identity/v3/projects/6add1594e65e475d8d6e4b6609c5046f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:04.157305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e5887-998d-4fc7-8b69-54c407b6f481 service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:04.168877 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e5887-998d-4fc7-8b69-54c407b6f481 service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:04.238733 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e1312-7003-4eac-b881-aea4890d4935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:04.239379 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e1312-7003-4eac-b881-aea4890d4935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:04.239650 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e1312-7003-4eac-b881-aea4890d4935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:04.239952 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e1312-7003-4eac-b881-aea4890d4935 admin admin] PATH_INFO: `/v3/users/7c7477061e6f47a8a71e0d38459dd472` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:04.241094 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e1312-7003-4eac-b881-aea4890d4935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c7477061e6f47a8a71e0d38459dd472)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:04.243425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e1312-7003-4eac-b881-aea4890d4935 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:04.246156 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:04.305028 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-387e1312-7003-4eac-b881-aea4890d4935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c7477061e6f47a8a71e0d38459dd472'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.322019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-387e1312-7003-4eac-b881-aea4890d4935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c7477061e6f47a8a71e0d38459dd472'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.329720 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-387e1312-7003-4eac-b881-aea4890d4935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c7477061e6f47a8a71e0d38459dd472'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.338913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-387e1312-7003-4eac-b881-aea4890d4935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c7477061e6f47a8a71e0d38459dd472'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.339624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-387e1312-7003-4eac-b881-aea4890d4935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c7477061e6f47a8a71e0d38459dd472'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.347541 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1386/2795] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:04 2022] DELETE /identity/v3/users/7c7477061e6f47a8a71e0d38459dd472 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:04.672980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:04.674588 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:04.706144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdd62e-fc75-4bbc-989b-8090246f8513 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:04.789552 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df54702-4783-4df5-b7fe-b05486ba433a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:04.790597 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df54702-4783-4df5-b7fe-b05486ba433a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:04.791058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df54702-4783-4df5-b7fe-b05486ba433a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:04.791518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df54702-4783-4df5-b7fe-b05486ba433a admin admin] PATH_INFO: `/v3/projects/5a883dfcf6744f07ac3759a9568baf98` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:04.793463 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df54702-4783-4df5-b7fe-b05486ba433a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a883dfcf6744f07ac3759a9568baf98)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:04.797081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df54702-4783-4df5-b7fe-b05486ba433a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:04.883982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4df54702-4783-4df5-b7fe-b05486ba433a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a883dfcf6744f07ac3759a9568baf98', 'request_id': 'req-4df54702-4783-4df5-b7fe-b05486ba433a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.890592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4df54702-4783-4df5-b7fe-b05486ba433a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a883dfcf6744f07ac3759a9568baf98', 'request_id': 'req-4df54702-4783-4df5-b7fe-b05486ba433a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.898853 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4df54702-4783-4df5-b7fe-b05486ba433a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a883dfcf6744f07ac3759a9568baf98', 'request_id': 'req-4df54702-4783-4df5-b7fe-b05486ba433a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.928052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4df54702-4783-4df5-b7fe-b05486ba433a admin admin] The token cache is being invalidate because project 5a883dfcf6744f07ac3759a9568baf98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:04.928409 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4df54702-4783-4df5-b7fe-b05486ba433a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.929288 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4df54702-4783-4df5-b7fe-b05486ba433a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:04.931104 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1410/2796] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:04 2022] DELETE /identity/v3/projects/5a883dfcf6744f07ac3759a9568baf98 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:04.936466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372a17f-b04a-4aed-9397-c0efe3836866 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:04.936772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372a17f-b04a-4aed-9397-c0efe3836866 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:04.937047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372a17f-b04a-4aed-9397-c0efe3836866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:04.958721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-0372a17f-b04a-4aed-9397-c0efe3836866 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:04.960048 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0372a17f-b04a-4aed-9397-c0efe3836866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:05.035380 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1387/2797] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:05.044257 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873c121-588b-4e4c-bf22-6783e77a3806 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:05.044606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873c121-588b-4e4c-bf22-6783e77a3806 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:05.045016 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873c121-588b-4e4c-bf22-6783e77a3806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:05.066886 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4873c121-588b-4e4c-bf22-6783e77a3806 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:05.068180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4873c121-588b-4e4c-bf22-6783e77a3806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:05.077567 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1411/2798] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:05.082482 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372a17f-b04a-4aed-9397-c0efe3836866 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:05.093624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372a17f-b04a-4aed-9397-c0efe3836866 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:05.099464 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a5821-7b25-4292-8aa6-f8ece30d1633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:05.100224 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a5821-7b25-4292-8aa6-f8ece30d1633 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:05.100517 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a5821-7b25-4292-8aa6-f8ece30d1633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:05.100795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a5821-7b25-4292-8aa6-f8ece30d1633 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:05.101740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a5821-7b25-4292-8aa6-f8ece30d1633 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:44:05.101740 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a5821-7b25-4292-8aa6-f8ece30d1633 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:05.103354 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a5821-7b25-4292-8aa6-f8ece30d1633 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:05.118310 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1388/2799] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:44:05.123015 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873c121-588b-4e4c-bf22-6783e77a3806 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:05.133563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873c121-588b-4e4c-bf22-6783e77a3806 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:05.138700 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41431d-d9d1-46cf-9a5c-3b1a43e03e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:05.139326 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41431d-d9d1-46cf-9a5c-3b1a43e03e8b admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:05.139608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41431d-d9d1-46cf-9a5c-3b1a43e03e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:05.139911 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41431d-d9d1-46cf-9a5c-3b1a43e03e8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:05.140569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41431d-d9d1-46cf-9a5c-3b1a43e03e8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:05.142507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41431d-d9d1-46cf-9a5c-3b1a43e03e8b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:05.178806 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1412/2800] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:05 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:05.180559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a5821-7b25-4292-8aa6-f8ece30d1633 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:05.191211 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a5821-7b25-4292-8aa6-f8ece30d1633 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:05.267001 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a33ae-0db2-4ce7-828f-2a26776a09a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:05.267622 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a33ae-0db2-4ce7-828f-2a26776a09a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:05.267920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a33ae-0db2-4ce7-828f-2a26776a09a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:05.268189 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a33ae-0db2-4ce7-828f-2a26776a09a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:05.268825 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a33ae-0db2-4ce7-828f-2a26776a09a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:05.270784 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a33ae-0db2-4ce7-828f-2a26776a09a7 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:05.297554 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:05.304865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1389/2801] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:05 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:44:05.309232 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41431d-d9d1-46cf-9a5c-3b1a43e03e8b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:05.321428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41431d-d9d1-46cf-9a5c-3b1a43e03e8b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:05.327114 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2455b-dc94-4d31-8bd4-539d33a59892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:05.328153 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2455b-dc94-4d31-8bd4-539d33a59892 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:05.328616 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2455b-dc94-4d31-8bd4-539d33a59892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:05.329062 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2455b-dc94-4d31-8bd4-539d33a59892 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:05.330146 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2455b-dc94-4d31-8bd4-539d33a59892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:05.332741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2455b-dc94-4d31-8bd4-539d33a59892 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:05.349094 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1413/2802] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:05.355474 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a33ae-0db2-4ce7-828f-2a26776a09a7 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:05.366844 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a33ae-0db2-4ce7-828f-2a26776a09a7 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:05.371832 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb5749-01b3-434c-b748-24089c607c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:05.372438 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb5749-01b3-434c-b748-24089c607c62 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:05.372714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb5749-01b3-434c-b748-24089c607c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:05.373010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb5749-01b3-434c-b748-24089c607c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:05.373582 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb5749-01b3-434c-b748-24089c607c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:05.375184 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb5749-01b3-434c-b748-24089c607c62 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:05.386927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1390/2803] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:05.392650 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2455b-dc94-4d31-8bd4-539d33a59892 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:05.408702 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2455b-dc94-4d31-8bd4-539d33a59892 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:05.414453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e12bcb-4635-444b-83c8-90d29b9e0ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:05.415263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e12bcb-4635-444b-83c8-90d29b9e0ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:05.415532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e12bcb-4635-444b-83c8-90d29b9e0ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:05.415805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e12bcb-4635-444b-83c8-90d29b9e0ca2 admin admin] PATH_INFO: `/v3/projects/23761d92298d4d1997d0e4cb5a5c6b25/users/e51c93cab1ea4b76811b6830b1240fdf/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:05.438538 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e12bcb-4635-444b-83c8-90d29b9e0ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23761d92298d4d1997d0e4cb5a5c6b25, user_id=e51c93cab1ea4b76811b6830b1240fdf, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:05.441678 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e12bcb-4635-444b-83c8-90d29b9e0ca2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:05.458839 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1414/2804] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:05 2022] PUT /identity/v3/projects/23761d92298d4d1997d0e4cb5a5c6b25/users/e51c93cab1ea4b76811b6830b1240fdf/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:05.471479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8dfc1-f717-4550-922c-e704e2018efa None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:05.471780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8dfc1-f717-4550-922c-e704e2018efa None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:05.472086 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8dfc1-f717-4550-922c-e704e2018efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:05.505762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-47d8dfc1-f717-4550-922c-e704e2018efa None None] MFA Rules not processed for user `e51c93cab1ea4b76811b6830b1240fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:05.507072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-47d8dfc1-f717-4550-922c-e704e2018efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:05.590075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1391/2805] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:05.604814 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e12bcb-4635-444b-83c8-90d29b9e0ca2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:05.607356 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-09e12bcb-4635-444b-83c8-90d29b9e0ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:05.641084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:05.677917 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e12bcb-4635-444b-83c8-90d29b9e0ca2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:05.682789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681df1c-5001-4e57-acdf-d6feab3728c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:05.683476 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681df1c-5001-4e57-acdf-d6feab3728c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:05.683772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681df1c-5001-4e57-acdf-d6feab3728c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:05.684083 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681df1c-5001-4e57-acdf-d6feab3728c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:05.693745 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681df1c-5001-4e57-acdf-d6feab3728c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:05.696335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681df1c-5001-4e57-acdf-d6feab3728c4 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:05.708188 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1415/2806] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:44:08.456258 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8dfc1-f717-4550-922c-e704e2018efa None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:08.542622 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8dfc1-f717-4550-922c-e704e2018efa None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:08.548603 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06064e60-a495-464e-990e-36ff2981af89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:08.549732 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06064e60-a495-464e-990e-36ff2981af89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:08.550077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06064e60-a495-464e-990e-36ff2981af89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:08.550338 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06064e60-a495-464e-990e-36ff2981af89 admin admin] PATH_INFO: `/v3/users/e51c93cab1ea4b76811b6830b1240fdf` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:08.551686 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06064e60-a495-464e-990e-36ff2981af89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e51c93cab1ea4b76811b6830b1240fdf)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:08.554178 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06064e60-a495-464e-990e-36ff2981af89 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:08.621065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-06064e60-a495-464e-990e-36ff2981af89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e51c93cab1ea4b76811b6830b1240fdf'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:08.637737 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-06064e60-a495-464e-990e-36ff2981af89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e51c93cab1ea4b76811b6830b1240fdf'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:08.646339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-06064e60-a495-464e-990e-36ff2981af89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e51c93cab1ea4b76811b6830b1240fdf'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:08.651407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:08.656018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-06064e60-a495-464e-990e-36ff2981af89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e51c93cab1ea4b76811b6830b1240fdf'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:08.658451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-06064e60-a495-464e-990e-36ff2981af89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e51c93cab1ea4b76811b6830b1240fdf'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:08.667426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1392/2807] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:08 2022] DELETE /identity/v3/users/e51c93cab1ea4b76811b6830b1240fdf => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:08.672327 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5194f-0108-4860-906a-9c2d1224230b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:08.673322 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5194f-0108-4860-906a-9c2d1224230b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:08.673660 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5194f-0108-4860-906a-9c2d1224230b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:08.696157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f1c5194f-0108-4860-906a-9c2d1224230b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:08.697336 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f1c5194f-0108-4860-906a-9c2d1224230b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:08.780712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1416/2808] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:08.794240 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06064e60-a495-464e-990e-36ff2981af89 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:08.795785 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-06064e60-a495-464e-990e-36ff2981af89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:08.844786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5194f-0108-4860-906a-9c2d1224230b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:08.846497 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f1c5194f-0108-4860-906a-9c2d1224230b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:08.863282 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5194f-0108-4860-906a-9c2d1224230b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:08.875714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac742d-f330-42a1-bf58-4f8be139bd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:08.876785 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac742d-f330-42a1-bf58-4f8be139bd5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:08.877236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac742d-f330-42a1-bf58-4f8be139bd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:08.877507 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac742d-f330-42a1-bf58-4f8be139bd5d admin admin] PATH_INFO: `/v3/users/ebf5138b4bf84239a69919f3b49ef99e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:08.878772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac742d-f330-42a1-bf58-4f8be139bd5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebf5138b4bf84239a69919f3b49ef99e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:08.881454 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac742d-f330-42a1-bf58-4f8be139bd5d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:08.900114 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06064e60-a495-464e-990e-36ff2981af89 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:08.905051 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7318a-39b4-4288-adce-8336045dded3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:08.906384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7318a-39b4-4288-adce-8336045dded3 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:08.906765 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7318a-39b4-4288-adce-8336045dded3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:08.907136 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7318a-39b4-4288-adce-8336045dded3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:08.919951 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7318a-39b4-4288-adce-8336045dded3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:08.921728 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7318a-39b4-4288-adce-8336045dded3 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:08.933192 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1393/2809] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:08.946681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6dac742d-f330-42a1-bf58-4f8be139bd5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebf5138b4bf84239a69919f3b49ef99e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:08.958956 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6dac742d-f330-42a1-bf58-4f8be139bd5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebf5138b4bf84239a69919f3b49ef99e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:08.959676 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6dac742d-f330-42a1-bf58-4f8be139bd5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebf5138b4bf84239a69919f3b49ef99e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:08.982091 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6dac742d-f330-42a1-bf58-4f8be139bd5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebf5138b4bf84239a69919f3b49ef99e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:08.995326 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6dac742d-f330-42a1-bf58-4f8be139bd5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebf5138b4bf84239a69919f3b49ef99e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:09.004778 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1417/2810] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:08 2022] DELETE /identity/v3/users/ebf5138b4bf84239a69919f3b49ef99e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:09.010747 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f4c8c-27a1-43b9-83c6-4028f6d098f9 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:09.011069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f4c8c-27a1-43b9-83c6-4028f6d098f9 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:09.011342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f4c8c-27a1-43b9-83c6-4028f6d098f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:09.031724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-ff7f4c8c-27a1-43b9-83c6-4028f6d098f9 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:09.033088 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ff7f4c8c-27a1-43b9-83c6-4028f6d098f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:09.111523 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1394/2811] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:09.120362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac742d-f330-42a1-bf58-4f8be139bd5d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:09.121834 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6dac742d-f330-42a1-bf58-4f8be139bd5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:09.183994 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:09.204096 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac742d-f330-42a1-bf58-4f8be139bd5d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:09.212982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb9dcb-3cae-46e8-bc6b-c6e767260cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:09.214418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb9dcb-3cae-46e8-bc6b-c6e767260cad service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:09.214879 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb9dcb-3cae-46e8-bc6b-c6e767260cad service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:09.215236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb9dcb-3cae-46e8-bc6b-c6e767260cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:09.232720 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb9dcb-3cae-46e8-bc6b-c6e767260cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:09.234751 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb9dcb-3cae-46e8-bc6b-c6e767260cad service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:09.248913 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1418/2812] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:09.258865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f4c8c-27a1-43b9-83c6-4028f6d098f9 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:09.263841 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ff7f4c8c-27a1-43b9-83c6-4028f6d098f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:09.299341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f4c8c-27a1-43b9-83c6-4028f6d098f9 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:09.307019 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:09.308018 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:09.308612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:09.308917 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] PATH_INFO: `/v3/projects/23761d92298d4d1997d0e4cb5a5c6b25` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:09.310900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23761d92298d4d1997d0e4cb5a5c6b25)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:09.314283 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:09.392913 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23761d92298d4d1997d0e4cb5a5c6b25', 'request_id': 'req-fc726c67-e81d-43a7-916f-1bb7371da232'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:09.409577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23761d92298d4d1997d0e4cb5a5c6b25', 'request_id': 'req-fc726c67-e81d-43a7-916f-1bb7371da232'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:09.423492 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23761d92298d4d1997d0e4cb5a5c6b25', 'request_id': 'req-fc726c67-e81d-43a7-916f-1bb7371da232'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:09.438204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] The token cache is being invalidate because project 23761d92298d4d1997d0e4cb5a5c6b25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:09.438555 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:09.439175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:09.441147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1395/2813] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:09 2022] DELETE /identity/v3/projects/23761d92298d4d1997d0e4cb5a5c6b25 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:09.446606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6078e-6817-4e5a-8d0d-3f7d35bf6a42 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:09.446893 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6078e-6817-4e5a-8d0d-3f7d35bf6a42 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:09.447181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6078e-6817-4e5a-8d0d-3f7d35bf6a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:09.468603 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3ef6078e-6817-4e5a-8d0d-3f7d35bf6a42 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:09.469787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3ef6078e-6817-4e5a-8d0d-3f7d35bf6a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:09.548708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1419/2814] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:09.592880 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:09.594542 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:09.619135 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc726c67-e81d-43a7-916f-1bb7371da232 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:09.626847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196f72d-2696-4a0c-8316-da428a09573f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:09.627800 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196f72d-2696-4a0c-8316-da428a09573f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:09.628285 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196f72d-2696-4a0c-8316-da428a09573f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:09.628553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196f72d-2696-4a0c-8316-da428a09573f admin admin] PATH_INFO: `/v3/projects/0410ff3e18434c5e8a2e7b33587c487c` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:09.629809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196f72d-2696-4a0c-8316-da428a09573f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0410ff3e18434c5e8a2e7b33587c487c)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:09.632600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196f72d-2696-4a0c-8316-da428a09573f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:09.634335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979bedc-3dfb-4b97-b194-3051febf6383 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:09.634712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979bedc-3dfb-4b97-b194-3051febf6383 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:09.634992 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979bedc-3dfb-4b97-b194-3051febf6383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:09.655384 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:09.656518 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1979bedc-3dfb-4b97-b194-3051febf6383 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:09.657895 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1979bedc-3dfb-4b97-b194-3051febf6383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:09.671941 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1420/2815] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:09.673489 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979bedc-3dfb-4b97-b194-3051febf6383 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:09.683071 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979bedc-3dfb-4b97-b194-3051febf6383 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:09.694945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3196f72d-2696-4a0c-8316-da428a09573f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0410ff3e18434c5e8a2e7b33587c487c', 'request_id': 'req-3196f72d-2696-4a0c-8316-da428a09573f'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:09.704768 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3196f72d-2696-4a0c-8316-da428a09573f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0410ff3e18434c5e8a2e7b33587c487c', 'request_id': 'req-3196f72d-2696-4a0c-8316-da428a09573f'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:09.711468 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3196f72d-2696-4a0c-8316-da428a09573f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0410ff3e18434c5e8a2e7b33587c487c', 'request_id': 'req-3196f72d-2696-4a0c-8316-da428a09573f'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:09.724642 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3196f72d-2696-4a0c-8316-da428a09573f admin admin] The token cache is being invalidate because project 0410ff3e18434c5e8a2e7b33587c487c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:09.724994 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3196f72d-2696-4a0c-8316-da428a09573f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:09.726118 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-3196f72d-2696-4a0c-8316-da428a09573f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:09.727850 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1396/2816] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:09 2022] DELETE /identity/v3/projects/0410ff3e18434c5e8a2e7b33587c487c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:09.734127 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9f680-2576-498f-b925-2a4384a6ee90 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:09.734588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9f680-2576-498f-b925-2a4384a6ee90 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:09.735054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9f680-2576-498f-b925-2a4384a6ee90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:09.747480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d9cc7-1127-4183-802b-6728e41312cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:09.748655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d9cc7-1127-4183-802b-6728e41312cd admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:09.748655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d9cc7-1127-4183-802b-6728e41312cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:09.748655 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d9cc7-1127-4183-802b-6728e41312cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:09.749335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d9cc7-1127-4183-802b-6728e41312cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:44:09.749695 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d9cc7-1127-4183-802b-6728e41312cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:09.751321 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d9cc7-1127-4183-802b-6728e41312cd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:09.763201 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-83c9f680-2576-498f-b925-2a4384a6ee90 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:09.764440 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-83c9f680-2576-498f-b925-2a4384a6ee90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:09.764964 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1421/2817] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:44:09.770291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d9cc7-1127-4183-802b-6728e41312cd admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:09.771968 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-073d9cc7-1127-4183-802b-6728e41312cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:09.773287 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1397/2818] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:09.784536 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4946d5b-ffb6-4b6d-83eb-572f1a049fb9 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:09.784955 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4946d5b-ffb6-4b6d-83eb-572f1a049fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:09.785433 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4946d5b-ffb6-4b6d-83eb-572f1a049fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:09.786180 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d9cc7-1127-4183-802b-6728e41312cd admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:09.792434 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd133b57-e6ac-4601-bb5d-5fac9c675c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:09.793103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133b57-e6ac-4601-bb5d-5fac9c675c8d admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:09.793376 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133b57-e6ac-4601-bb5d-5fac9c675c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:09.793712 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133b57-e6ac-4601-bb5d-5fac9c675c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:09.794422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd133b57-e6ac-4601-bb5d-5fac9c675c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:09.796720 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd133b57-e6ac-4601-bb5d-5fac9c675c8d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:09.810878 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-b4946d5b-ffb6-4b6d-83eb-572f1a049fb9 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:09.812136 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b4946d5b-ffb6-4b6d-83eb-572f1a049fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:09.823345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1398/2819] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:09.827559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4946d5b-ffb6-4b6d-83eb-572f1a049fb9 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:09.828677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1422/2820] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:09 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:44:09.832532 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd133b57-e6ac-4601-bb5d-5fac9c675c8d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:09.837049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4946d5b-ffb6-4b6d-83eb-572f1a049fb9 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:09.889444 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:09.900445 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04e3ce-9b58-408e-a0a9-82f26a98ac76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:09.901063 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04e3ce-9b58-408e-a0a9-82f26a98ac76 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:09.901345 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04e3ce-9b58-408e-a0a9-82f26a98ac76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:09.901610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04e3ce-9b58-408e-a0a9-82f26a98ac76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:09.902650 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04e3ce-9b58-408e-a0a9-82f26a98ac76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:44:09.902650 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04e3ce-9b58-408e-a0a9-82f26a98ac76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:09.904236 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04e3ce-9b58-408e-a0a9-82f26a98ac76 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:09.906942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd133b57-e6ac-4601-bb5d-5fac9c675c8d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:09.912833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30583e9b-d941-4dfb-8892-3784b98213a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:09.913881 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30583e9b-d941-4dfb-8892-3784b98213a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:09.914396 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30583e9b-d941-4dfb-8892-3784b98213a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:09.915482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30583e9b-d941-4dfb-8892-3784b98213a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:09.915482 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30583e9b-d941-4dfb-8892-3784b98213a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:09.917763 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30583e9b-d941-4dfb-8892-3784b98213a4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:09.940156 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1399/2821] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:44:09.940156 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04e3ce-9b58-408e-a0a9-82f26a98ac76 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:09.941414 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04e3ce-9b58-408e-a0a9-82f26a98ac76 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:09.944587 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1423/2822] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:09 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:09.948280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547aa2a-aef9-4072-9594-2ef0f9b684dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:09.949040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547aa2a-aef9-4072-9594-2ef0f9b684dd admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:09.949406 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547aa2a-aef9-4072-9594-2ef0f9b684dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:09.950314 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30583e9b-d941-4dfb-8892-3784b98213a4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:09.950462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547aa2a-aef9-4072-9594-2ef0f9b684dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:09.950462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547aa2a-aef9-4072-9594-2ef0f9b684dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:09.952821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547aa2a-aef9-4072-9594-2ef0f9b684dd admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:09.967007 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30583e9b-d941-4dfb-8892-3784b98213a4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:09.975018 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40115603-c5a9-4b19-9bfe-7744ea539e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:09.975791 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40115603-c5a9-4b19-9bfe-7744ea539e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:09.976219 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40115603-c5a9-4b19-9bfe-7744ea539e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:09.976684 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40115603-c5a9-4b19-9bfe-7744ea539e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:09.977829 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40115603-c5a9-4b19-9bfe-7744ea539e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:09.980352 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40115603-c5a9-4b19-9bfe-7744ea539e63 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:09.995749 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1400/2823] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:09 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:09.997432 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547aa2a-aef9-4072-9594-2ef0f9b684dd admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:09.997775 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1424/2824] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:44:10.003459 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40115603-c5a9-4b19-9bfe-7744ea539e63 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:10.020508 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40115603-c5a9-4b19-9bfe-7744ea539e63 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:10.088529 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547aa2a-aef9-4072-9594-2ef0f9b684dd admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:10.095073 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18712fa7-b4e0-40e9-94ef-c15e02b37213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:10.095964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18712fa7-b4e0-40e9-94ef-c15e02b37213 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:10.096593 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18712fa7-b4e0-40e9-94ef-c15e02b37213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:10.096869 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18712fa7-b4e0-40e9-94ef-c15e02b37213 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:10.097513 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18712fa7-b4e0-40e9-94ef-c15e02b37213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:10.100071 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18712fa7-b4e0-40e9-94ef-c15e02b37213 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:10.113061 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:10.118691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c79510-7c58-498f-8a37-59aa7637b77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:10.119428 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c79510-7c58-498f-8a37-59aa7637b77e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:10.119722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c79510-7c58-498f-8a37-59aa7637b77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:10.120084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c79510-7c58-498f-8a37-59aa7637b77e admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:10.120754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c79510-7c58-498f-8a37-59aa7637b77e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:10.122614 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c79510-7c58-498f-8a37-59aa7637b77e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:10.132998 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1401/2825] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:09 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:44:10.135295 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18712fa7-b4e0-40e9-94ef-c15e02b37213 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:10.143265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1425/2826] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:10.149617 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c79510-7c58-498f-8a37-59aa7637b77e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:10.153425 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18712fa7-b4e0-40e9-94ef-c15e02b37213 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:10.160892 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c79510-7c58-498f-8a37-59aa7637b77e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:10.166322 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba3385-266d-4fd7-b14c-c1921549eae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:10.166968 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba3385-266d-4fd7-b14c-c1921549eae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:10.167239 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba3385-266d-4fd7-b14c-c1921549eae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:10.167486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba3385-266d-4fd7-b14c-c1921549eae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:10.168186 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba3385-266d-4fd7-b14c-c1921549eae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:10.169914 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba3385-266d-4fd7-b14c-c1921549eae8 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:10.170237 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e84d7-b1d2-499d-bb49-455c4463ca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:10.170722 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e84d7-b1d2-499d-bb49-455c4463ca87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:10.170964 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e84d7-b1d2-499d-bb49-455c4463ca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:10.171528 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e84d7-b1d2-499d-bb49-455c4463ca87 admin admin] PATH_INFO: `/v3/projects/ec4d86f1d7bc4cc9a39ff55d4dc6a801/users/ed4710f614cc431683a9c2a4f3d9d7b6/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:10.191052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e84d7-b1d2-499d-bb49-455c4463ca87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec4d86f1d7bc4cc9a39ff55d4dc6a801, user_id=ed4710f614cc431683a9c2a4f3d9d7b6, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:10.193217 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1402/2827] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:44:10.196986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e84d7-b1d2-499d-bb49-455c4463ca87 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:10.200466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba3385-266d-4fd7-b14c-c1921549eae8 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:10.215169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba3385-266d-4fd7-b14c-c1921549eae8 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:10.220464 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1426/2828] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:10 2022] PUT /identity/v3/projects/ec4d86f1d7bc4cc9a39ff55d4dc6a801/users/ed4710f614cc431683a9c2a4f3d9d7b6/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:10.226954 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999a8c4-5db3-4db9-830c-06c0979af9c1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:10.226954 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999a8c4-5db3-4db9-830c-06c0979af9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:10.227301 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999a8c4-5db3-4db9-830c-06c0979af9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:10.252599 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0999a8c4-5db3-4db9-830c-06c0979af9c1 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:10.254560 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0999a8c4-5db3-4db9-830c-06c0979af9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:10.304020 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb526b-75d6-4644-a381-f6a5106ffead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:10.304671 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb526b-75d6-4644-a381-f6a5106ffead admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:10.305015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb526b-75d6-4644-a381-f6a5106ffead admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:10.305348 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb526b-75d6-4644-a381-f6a5106ffead admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:10.306054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb526b-75d6-4644-a381-f6a5106ffead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:10.310670 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb526b-75d6-4644-a381-f6a5106ffead admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:10.331610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1403/2829] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:10.332687 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb526b-75d6-4644-a381-f6a5106ffead admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:10.344268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb526b-75d6-4644-a381-f6a5106ffead admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:10.352419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1427/2830] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:10.352669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238ad43-effc-4b5c-8e8c-e0a4cdd542c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:10.352669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238ad43-effc-4b5c-8e8c-e0a4cdd542c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:10.352669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238ad43-effc-4b5c-8e8c-e0a4cdd542c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:10.352669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238ad43-effc-4b5c-8e8c-e0a4cdd542c3 admin admin] PATH_INFO: `/v3/projects/be84b1cd7ec7498097d73bafe8a58ebe/users/242b69b3a3454079a9256b86dd539a68/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:10.359810 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999a8c4-5db3-4db9-830c-06c0979af9c1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:10.361538 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0999a8c4-5db3-4db9-830c-06c0979af9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:10.398939 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238ad43-effc-4b5c-8e8c-e0a4cdd542c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be84b1cd7ec7498097d73bafe8a58ebe, user_id=242b69b3a3454079a9256b86dd539a68, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:10.402182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238ad43-effc-4b5c-8e8c-e0a4cdd542c3 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:10.419541 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1404/2831] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:10 2022] PUT /identity/v3/projects/be84b1cd7ec7498097d73bafe8a58ebe/users/242b69b3a3454079a9256b86dd539a68/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:10.431933 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e858749-9914-4b04-aaee-5b3187c98133 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:10.432485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e858749-9914-4b04-aaee-5b3187c98133 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:10.432735 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e858749-9914-4b04-aaee-5b3187c98133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:10.435998 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:10.461152 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999a8c4-5db3-4db9-830c-06c0979af9c1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:10.471905 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-2e858749-9914-4b04-aaee-5b3187c98133 None None] MFA Rules not processed for user `242b69b3a3454079a9256b86dd539a68`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:10.473217 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2e858749-9914-4b04-aaee-5b3187c98133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:10.512517 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b7699-241f-423e-967e-f02b54b9a87b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:10.513053 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b7699-241f-423e-967e-f02b54b9a87b service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:10.513384 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b7699-241f-423e-967e-f02b54b9a87b service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:10.513743 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b7699-241f-423e-967e-f02b54b9a87b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:10.559722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1405/2832] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:10.565024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e858749-9914-4b04-aaee-5b3187c98133 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:10.596048 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b7699-241f-423e-967e-f02b54b9a87b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:10.598462 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b7699-241f-423e-967e-f02b54b9a87b service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:10.609677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1428/2833] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:44:10.625059 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:10.662903 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e858749-9914-4b04-aaee-5b3187c98133 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:10.669174 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676066e6-904e-4841-8774-c3c31098cc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:10.669817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676066e6-904e-4841-8774-c3c31098cc7c admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:10.670098 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676066e6-904e-4841-8774-c3c31098cc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:10.670409 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676066e6-904e-4841-8774-c3c31098cc7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:10.671015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676066e6-904e-4841-8774-c3c31098cc7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:10.673007 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676066e6-904e-4841-8774-c3c31098cc7c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:10.706352 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1406/2834] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:10 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:10.711826 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b7699-241f-423e-967e-f02b54b9a87b service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:10.723951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b7699-241f-423e-967e-f02b54b9a87b service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:10.821042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf5e41-da58-4c3c-9713-e79f54f13a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:10.821906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf5e41-da58-4c3c-9713-e79f54f13a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:10.822264 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf5e41-da58-4c3c-9713-e79f54f13a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:10.822662 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf5e41-da58-4c3c-9713-e79f54f13a0e admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:10.823418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf5e41-da58-4c3c-9713-e79f54f13a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:10.825667 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf5e41-da58-4c3c-9713-e79f54f13a0e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:10.848564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1429/2835] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:10 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:10.853654 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676066e6-904e-4841-8774-c3c31098cc7c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:10.867157 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676066e6-904e-4841-8774-c3c31098cc7c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:10.873422 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fccf1-f7e7-45fe-8a68-fe922e377c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:10.874042 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fccf1-f7e7-45fe-8a68-fe922e377c02 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:10.874504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fccf1-f7e7-45fe-8a68-fe922e377c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:10.874775 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fccf1-f7e7-45fe-8a68-fe922e377c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:10.875417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fccf1-f7e7-45fe-8a68-fe922e377c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:10.877076 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fccf1-f7e7-45fe-8a68-fe922e377c02 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:10.889225 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1407/2836] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:44:10.892583 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf5e41-da58-4c3c-9713-e79f54f13a0e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:10.902909 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf5e41-da58-4c3c-9713-e79f54f13a0e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:10.907857 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87cf908-1f18-48c3-8510-c863404c323a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:10.908498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cf908-1f18-48c3-8510-c863404c323a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:10.908825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cf908-1f18-48c3-8510-c863404c323a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:10.909125 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cf908-1f18-48c3-8510-c863404c323a admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:10.909951 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87cf908-1f18-48c3-8510-c863404c323a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:10.911842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87cf908-1f18-48c3-8510-c863404c323a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:10.927020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1430/2837] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:44:10.928549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fccf1-f7e7-45fe-8a68-fe922e377c02 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:10.940376 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fccf1-f7e7-45fe-8a68-fe922e377c02 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:10.946249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e35a9-2cd0-4b4c-b702-0c8fa76d7d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:10.947068 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e35a9-2cd0-4b4c-b702-0c8fa76d7d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:10.947404 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e35a9-2cd0-4b4c-b702-0c8fa76d7d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:10.947702 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e35a9-2cd0-4b4c-b702-0c8fa76d7d05 admin admin] PATH_INFO: `/v3/projects/b27e45b465134ef3bbb0a77d9ff59577/users/124e6c23836d4d828b143f2305cd5984/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:10.965106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e35a9-2cd0-4b4c-b702-0c8fa76d7d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27e45b465134ef3bbb0a77d9ff59577, user_id=124e6c23836d4d828b143f2305cd5984, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:10.968056 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e35a9-2cd0-4b4c-b702-0c8fa76d7d05 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:11.011932 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1408/2838] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:10 2022] PUT /identity/v3/projects/b27e45b465134ef3bbb0a77d9ff59577/users/124e6c23836d4d828b143f2305cd5984/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:11.069730 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a0f0b-6eb0-450f-9edb-845674aea708 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:11.070079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a0f0b-6eb0-450f-9edb-845674aea708 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:11.070676 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a0f0b-6eb0-450f-9edb-845674aea708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:11.120981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-523a0f0b-6eb0-450f-9edb-845674aea708 None None] MFA Rules not processed for user `124e6c23836d4d828b143f2305cd5984`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:11.123190 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-523a0f0b-6eb0-450f-9edb-845674aea708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:11.144400 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:11.208290 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1431/2839] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:11.213849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e35a9-2cd0-4b4c-b702-0c8fa76d7d05 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:11.287641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e35a9-2cd0-4b4c-b702-0c8fa76d7d05 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:11.293624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886a1ec-73fe-4312-9f85-3ce526276d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:11.294295 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886a1ec-73fe-4312-9f85-3ce526276d05 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:11.294543 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886a1ec-73fe-4312-9f85-3ce526276d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:11.294808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886a1ec-73fe-4312-9f85-3ce526276d05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:11.295449 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886a1ec-73fe-4312-9f85-3ce526276d05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:11.297404 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886a1ec-73fe-4312-9f85-3ce526276d05 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:11.334357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1409/2840] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:11 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:44:11.337240 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a0f0b-6eb0-450f-9edb-845674aea708 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:11.362914 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a0f0b-6eb0-450f-9edb-845674aea708 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:11.463868 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c66b8-937e-4fe6-b85a-457ddf46867b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:11.464693 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c66b8-937e-4fe6-b85a-457ddf46867b admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:11.465042 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c66b8-937e-4fe6-b85a-457ddf46867b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:11.465496 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c66b8-937e-4fe6-b85a-457ddf46867b admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:11.466603 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c66b8-937e-4fe6-b85a-457ddf46867b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:11.468517 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c66b8-937e-4fe6-b85a-457ddf46867b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:11.504821 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1432/2841] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:11 2022] POST /identity/v3/users => generated 295 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:11.511875 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886a1ec-73fe-4312-9f85-3ce526276d05 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:11.526588 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886a1ec-73fe-4312-9f85-3ce526276d05 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:11.533144 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058e12a-ecd5-418c-8a01-183aae6db48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:11.533772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058e12a-ecd5-418c-8a01-183aae6db48c admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:11.534048 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058e12a-ecd5-418c-8a01-183aae6db48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:11.534318 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058e12a-ecd5-418c-8a01-183aae6db48c admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:11.534966 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7058e12a-ecd5-418c-8a01-183aae6db48c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:11.536698 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7058e12a-ecd5-418c-8a01-183aae6db48c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:11.551931 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1410/2842] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:11.555777 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c66b8-937e-4fe6-b85a-457ddf46867b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:11.572715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c66b8-937e-4fe6-b85a-457ddf46867b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:11.578052 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeba0d8-b47f-46bf-a807-a833790ebd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:11.579006 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba0d8-b47f-46bf-a807-a833790ebd7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:11.579399 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba0d8-b47f-46bf-a807-a833790ebd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:11.579710 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba0d8-b47f-46bf-a807-a833790ebd7e admin admin] PATH_INFO: `/v3/projects/3c7f6821015d4f53aa912c89b5fdeb1a/users/2ca46ba1a656436d9fe208584473d04b/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:11.600772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeba0d8-b47f-46bf-a807-a833790ebd7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7f6821015d4f53aa912c89b5fdeb1a, user_id=2ca46ba1a656436d9fe208584473d04b, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:11.604121 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeba0d8-b47f-46bf-a807-a833790ebd7e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:11.623147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1433/2843] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:11 2022] PUT /identity/v3/projects/3c7f6821015d4f53aa912c89b5fdeb1a/users/2ca46ba1a656436d9fe208584473d04b/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:11.627873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058e12a-ecd5-418c-8a01-183aae6db48c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:11.642546 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058e12a-ecd5-418c-8a01-183aae6db48c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:11.717498 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df5f10-b176-49c0-bbd7-1afede1baf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:11.718725 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df5f10-b176-49c0-bbd7-1afede1baf25 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:11.719160 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df5f10-b176-49c0-bbd7-1afede1baf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:11.719619 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df5f10-b176-49c0-bbd7-1afede1baf25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:11.720486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df5f10-b176-49c0-bbd7-1afede1baf25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:11.722278 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df5f10-b176-49c0-bbd7-1afede1baf25 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:11.736591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1411/2844] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:44:11.741123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeba0d8-b47f-46bf-a807-a833790ebd7e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:11.758155 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeba0d8-b47f-46bf-a807-a833790ebd7e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:11.768129 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148f9d7-74b3-49fe-b97f-68a6c3beae51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:11.768786 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148f9d7-74b3-49fe-b97f-68a6c3beae51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:11.769086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148f9d7-74b3-49fe-b97f-68a6c3beae51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:11.769356 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148f9d7-74b3-49fe-b97f-68a6c3beae51 admin admin] PATH_INFO: `/v3/projects/3c7f6821015d4f53aa912c89b5fdeb1a/users/2ca46ba1a656436d9fe208584473d04b/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:11.771447 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148f9d7-74b3-49fe-b97f-68a6c3beae51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7f6821015d4f53aa912c89b5fdeb1a, user_id=2ca46ba1a656436d9fe208584473d04b, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:11.774672 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148f9d7-74b3-49fe-b97f-68a6c3beae51 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:11.788058 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1434/2845] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:11 2022] PUT /identity/v3/projects/3c7f6821015d4f53aa912c89b5fdeb1a/users/2ca46ba1a656436d9fe208584473d04b/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:11.797711 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cad142-686b-4437-b63d-1bf4f1b22391 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:11.798010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cad142-686b-4437-b63d-1bf4f1b22391 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:11.798248 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cad142-686b-4437-b63d-1bf4f1b22391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:11.807553 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:11.836106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-67cad142-686b-4437-b63d-1bf4f1b22391 None None] MFA Rules not processed for user `2ca46ba1a656436d9fe208584473d04b`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:11.837375 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-67cad142-686b-4437-b63d-1bf4f1b22391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:11.914549 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1412/2846] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:11.932606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148f9d7-74b3-49fe-b97f-68a6c3beae51 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:11.935569 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7148f9d7-74b3-49fe-b97f-68a6c3beae51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:12.029779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148f9d7-74b3-49fe-b97f-68a6c3beae51 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:12.034643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf591cd0-4826-414b-9ed7-c729e2467e0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:12.035359 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf591cd0-4826-414b-9ed7-c729e2467e0c service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:12.035759 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf591cd0-4826-414b-9ed7-c729e2467e0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:12.036214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf591cd0-4826-414b-9ed7-c729e2467e0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:12.075411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:12.144486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf591cd0-4826-414b-9ed7-c729e2467e0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:12.146811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf591cd0-4826-414b-9ed7-c729e2467e0c service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:12.162081 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1435/2847] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:14.004895 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cad142-686b-4437-b63d-1bf4f1b22391 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:14.008211 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-67cad142-686b-4437-b63d-1bf4f1b22391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:14.049014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cad142-686b-4437-b63d-1bf4f1b22391 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:14.056874 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb669e63-ccea-4022-828a-6b1332e8ee09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:14.057667 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb669e63-ccea-4022-828a-6b1332e8ee09 service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:14.058021 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb669e63-ccea-4022-828a-6b1332e8ee09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:14.058356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb669e63-ccea-4022-828a-6b1332e8ee09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:14.167868 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb669e63-ccea-4022-828a-6b1332e8ee09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:14.170049 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb669e63-ccea-4022-828a-6b1332e8ee09 service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:14.181849 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1413/2848] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:15.476950 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf591cd0-4826-414b-9ed7-c729e2467e0c service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:15.534048 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf591cd0-4826-414b-9ed7-c729e2467e0c service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:15.545839 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954aa46c-d536-44bf-ae2d-0d5426ba357a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:15.546955 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954aa46c-d536-44bf-ae2d-0d5426ba357a service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:15.548259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954aa46c-d536-44bf-ae2d-0d5426ba357a service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:15.548744 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954aa46c-d536-44bf-ae2d-0d5426ba357a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:15.592272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954aa46c-d536-44bf-ae2d-0d5426ba357a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:15.596429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954aa46c-d536-44bf-ae2d-0d5426ba357a service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:15.619367 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1436/2849] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:16.276199 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22b455-59ba-4a7b-b3a0-d6f3b82f965a None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:16.276808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22b455-59ba-4a7b-b3a0-d6f3b82f965a None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:16.276808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22b455-59ba-4a7b-b3a0-d6f3b82f965a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:16.324521 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-fc22b455-59ba-4a7b-b3a0-d6f3b82f965a None None] MFA Rules not processed for user `ed4710f614cc431683a9c2a4f3d9d7b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:16.326051 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-fc22b455-59ba-4a7b-b3a0-d6f3b82f965a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:16.414493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=100 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:16.444293 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1414/2850] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:16.450419 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954aa46c-d536-44bf-ae2d-0d5426ba357a service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:16.615147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954aa46c-d536-44bf-ae2d-0d5426ba357a service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:16.622265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ffe45-3500-4d38-b1f8-2889f8fa8266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:16.623307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ffe45-3500-4d38-b1f8-2889f8fa8266 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:16.623791 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ffe45-3500-4d38-b1f8-2889f8fa8266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:16.624207 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ffe45-3500-4d38-b1f8-2889f8fa8266 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:16.625099 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ffe45-3500-4d38-b1f8-2889f8fa8266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:16.627599 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ffe45-3500-4d38-b1f8-2889f8fa8266 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:16.685679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1437/2851] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:16 2022] POST /identity/v3/projects => generated 379 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:16.690330 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22b455-59ba-4a7b-b3a0-d6f3b82f965a None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:16.707014 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22b455-59ba-4a7b-b3a0-d6f3b82f965a None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:16.799793 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e073f6d-e89b-48e0-af72-7aa2aff6d730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:16.800992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e073f6d-e89b-48e0-af72-7aa2aff6d730 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:16.801290 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e073f6d-e89b-48e0-af72-7aa2aff6d730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:16.801563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e073f6d-e89b-48e0-af72-7aa2aff6d730 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:16.802242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e073f6d-e89b-48e0-af72-7aa2aff6d730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:16.807005 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e073f6d-e89b-48e0-af72-7aa2aff6d730 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:16.845408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1415/2852] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:16 2022] POST /identity/v3/users => generated 296 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:16.849106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ffe45-3500-4d38-b1f8-2889f8fa8266 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:16.866579 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ffe45-3500-4d38-b1f8-2889f8fa8266 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:16.879551 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d93c93-ff3b-44db-9701-6bef04fa13ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:16.880511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d93c93-ff3b-44db-9701-6bef04fa13ff admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:16.880787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d93c93-ff3b-44db-9701-6bef04fa13ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:16.881064 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d93c93-ff3b-44db-9701-6bef04fa13ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:16.882369 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d93c93-ff3b-44db-9701-6bef04fa13ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:16.884318 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d93c93-ff3b-44db-9701-6bef04fa13ff admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:16.909230 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1438/2853] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:16.910208 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e073f6d-e89b-48e0-af72-7aa2aff6d730 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:16.923865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e073f6d-e89b-48e0-af72-7aa2aff6d730 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:16.929763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91568f81-53b2-4030-a87a-c7facc1be700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:16.930888 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91568f81-53b2-4030-a87a-c7facc1be700 admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:16.931257 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91568f81-53b2-4030-a87a-c7facc1be700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:16.931602 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91568f81-53b2-4030-a87a-c7facc1be700 admin admin] PATH_INFO: `/v3/projects/0fcaa980a74a40ad940f0a53e18a77e5/users/9d52bc941fb2491091249353f1c677e5/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:16.958528 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91568f81-53b2-4030-a87a-c7facc1be700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fcaa980a74a40ad940f0a53e18a77e5, user_id=9d52bc941fb2491091249353f1c677e5, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:16.962222 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91568f81-53b2-4030-a87a-c7facc1be700 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:16.983579 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1416/2854] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:16 2022] PUT /identity/v3/projects/0fcaa980a74a40ad940f0a53e18a77e5/users/9d52bc941fb2491091249353f1c677e5/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:16.987566 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d93c93-ff3b-44db-9701-6bef04fa13ff admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:17.008551 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d93c93-ff3b-44db-9701-6bef04fa13ff admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:17.063572 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:17.100748 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffe406-82b9-44e5-9a4e-90e91904c980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:17.101480 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffe406-82b9-44e5-9a4e-90e91904c980 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:17.101779 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffe406-82b9-44e5-9a4e-90e91904c980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:17.102072 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffe406-82b9-44e5-9a4e-90e91904c980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:17.102867 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffe406-82b9-44e5-9a4e-90e91904c980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:17.104892 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffe406-82b9-44e5-9a4e-90e91904c980 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:17.117812 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1439/2855] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:17.122710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91568f81-53b2-4030-a87a-c7facc1be700 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:17.144698 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91568f81-53b2-4030-a87a-c7facc1be700 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:17.151378 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0e226-f55e-4f50-85af-588143688bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:17.152485 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0e226-f55e-4f50-85af-588143688bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:17.153065 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0e226-f55e-4f50-85af-588143688bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:17.153311 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0e226-f55e-4f50-85af-588143688bdf admin admin] PATH_INFO: `/v3/projects/0fcaa980a74a40ad940f0a53e18a77e5/users/9d52bc941fb2491091249353f1c677e5/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:17.156106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0e226-f55e-4f50-85af-588143688bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fcaa980a74a40ad940f0a53e18a77e5, user_id=9d52bc941fb2491091249353f1c677e5, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:17.160122 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0e226-f55e-4f50-85af-588143688bdf admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:17.187532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1417/2856] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:17 2022] PUT /identity/v3/projects/0fcaa980a74a40ad940f0a53e18a77e5/users/9d52bc941fb2491091249353f1c677e5/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:20.233808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffe406-82b9-44e5-9a4e-90e91904c980 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:20.250133 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffe406-82b9-44e5-9a4e-90e91904c980 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:20.357168 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333f2ad-8b2f-4560-83b4-2d3fddf6f818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:20.357977 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333f2ad-8b2f-4560-83b4-2d3fddf6f818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:20.358261 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333f2ad-8b2f-4560-83b4-2d3fddf6f818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:20.358531 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333f2ad-8b2f-4560-83b4-2d3fddf6f818 admin admin] PATH_INFO: `/v3/users/242b69b3a3454079a9256b86dd539a68` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:20.359732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333f2ad-8b2f-4560-83b4-2d3fddf6f818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=242b69b3a3454079a9256b86dd539a68)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:20.361714 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333f2ad-8b2f-4560-83b4-2d3fddf6f818 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:20.442998 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7333f2ad-8b2f-4560-83b4-2d3fddf6f818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '242b69b3a3454079a9256b86dd539a68'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:20.454329 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7333f2ad-8b2f-4560-83b4-2d3fddf6f818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '242b69b3a3454079a9256b86dd539a68'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:20.455715 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7333f2ad-8b2f-4560-83b4-2d3fddf6f818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '242b69b3a3454079a9256b86dd539a68'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:20.474175 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7333f2ad-8b2f-4560-83b4-2d3fddf6f818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '242b69b3a3454079a9256b86dd539a68'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:20.483976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7333f2ad-8b2f-4560-83b4-2d3fddf6f818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '242b69b3a3454079a9256b86dd539a68'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:20.501893 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1440/2857] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:20 2022] DELETE /identity/v3/users/242b69b3a3454079a9256b86dd539a68 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:20.504335 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa99b7-38f6-4ae8-8d1f-f16d832ae35e None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:20.504840 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa99b7-38f6-4ae8-8d1f-f16d832ae35e None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:20.505206 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa99b7-38f6-4ae8-8d1f-f16d832ae35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:20.575753 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-e1fa99b7-38f6-4ae8-8d1f-f16d832ae35e None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:20.577324 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e1fa99b7-38f6-4ae8-8d1f-f16d832ae35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:20.776204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1418/2858] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:20.782613 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333f2ad-8b2f-4560-83b4-2d3fddf6f818 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:20.784295 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7333f2ad-8b2f-4560-83b4-2d3fddf6f818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:20.925192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:20.956199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333f2ad-8b2f-4560-83b4-2d3fddf6f818 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:20.976013 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c7059-ad04-4de7-871f-ae98bf6e906d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:20.976805 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c7059-ad04-4de7-871f-ae98bf6e906d service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:20.977079 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c7059-ad04-4de7-871f-ae98bf6e906d service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:20.977345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c7059-ad04-4de7-871f-ae98bf6e906d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:21.020100 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c7059-ad04-4de7-871f-ae98bf6e906d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:21.021562 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c7059-ad04-4de7-871f-ae98bf6e906d service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:21.054901 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1441/2859] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:21.678820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa99b7-38f6-4ae8-8d1f-f16d832ae35e None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:21.682232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e1fa99b7-38f6-4ae8-8d1f-f16d832ae35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:21.697981 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa99b7-38f6-4ae8-8d1f-f16d832ae35e None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:21.704315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:21.705236 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:21.705824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:21.721568 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] PATH_INFO: `/v3/projects/be84b1cd7ec7498097d73bafe8a58ebe` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:21.723366 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be84b1cd7ec7498097d73bafe8a58ebe)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:21.726649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:21.812712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:21.844532 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be84b1cd7ec7498097d73bafe8a58ebe', 'request_id': 'req-cc9367d2-b8d5-4402-be58-679375f30fe8'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:21.858043 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be84b1cd7ec7498097d73bafe8a58ebe', 'request_id': 'req-cc9367d2-b8d5-4402-be58-679375f30fe8'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:21.878479 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be84b1cd7ec7498097d73bafe8a58ebe', 'request_id': 'req-cc9367d2-b8d5-4402-be58-679375f30fe8'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:21.899096 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] The token cache is being invalidate because project be84b1cd7ec7498097d73bafe8a58ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:21.899807 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:21.900789 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:21.909417 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1419/2860] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:21 2022] DELETE /identity/v3/projects/be84b1cd7ec7498097d73bafe8a58ebe => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:21.924440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c7059-ad04-4de7-871f-ae98bf6e906d service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:21.935008 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c7059-ad04-4de7-871f-ae98bf6e906d service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:22.054837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8fc3e-90d0-4006-8f02-a00b52e741ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:22.055688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8fc3e-90d0-4006-8f02-a00b52e741ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:22.056098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8fc3e-90d0-4006-8f02-a00b52e741ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:22.056488 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8fc3e-90d0-4006-8f02-a00b52e741ae admin admin] PATH_INFO: `/v3/users/124e6c23836d4d828b143f2305cd5984` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:22.057830 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8fc3e-90d0-4006-8f02-a00b52e741ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=124e6c23836d4d828b143f2305cd5984)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:22.060112 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8fc3e-90d0-4006-8f02-a00b52e741ae admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:22.148934 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-50d8fc3e-90d0-4006-8f02-a00b52e741ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '124e6c23836d4d828b143f2305cd5984'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:22.159875 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-50d8fc3e-90d0-4006-8f02-a00b52e741ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '124e6c23836d4d828b143f2305cd5984'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:22.161015 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-50d8fc3e-90d0-4006-8f02-a00b52e741ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '124e6c23836d4d828b143f2305cd5984'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:22.187236 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-50d8fc3e-90d0-4006-8f02-a00b52e741ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '124e6c23836d4d828b143f2305cd5984'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:22.198398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-50d8fc3e-90d0-4006-8f02-a00b52e741ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '124e6c23836d4d828b143f2305cd5984'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:22.210559 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1442/2861] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:21 2022] DELETE /identity/v3/users/124e6c23836d4d828b143f2305cd5984 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:22.716469 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:22.717010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:22.742539 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9367d2-b8d5-4402-be58-679375f30fe8 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:22.848787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:22.850275 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:22.850764 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:22.851260 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] PATH_INFO: `/v3/projects/b27e45b465134ef3bbb0a77d9ff59577` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:22.852809 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b27e45b465134ef3bbb0a77d9ff59577)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:22.854926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:22.945365 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b27e45b465134ef3bbb0a77d9ff59577', 'request_id': 'req-6e15dfdd-8431-4be1-8702-8a3f2c96a193'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:22.954973 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b27e45b465134ef3bbb0a77d9ff59577', 'request_id': 'req-6e15dfdd-8431-4be1-8702-8a3f2c96a193'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:22.970265 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b27e45b465134ef3bbb0a77d9ff59577', 'request_id': 'req-6e15dfdd-8431-4be1-8702-8a3f2c96a193'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:22.985277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] The token cache is being invalidate because project b27e45b465134ef3bbb0a77d9ff59577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:22.985277 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:22.985750 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:22.987250 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1420/2862] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:22 2022] DELETE /identity/v3/projects/b27e45b465134ef3bbb0a77d9ff59577 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:22.992111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8fc3e-90d0-4006-8f02-a00b52e741ae admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:22.993575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-50d8fc3e-90d0-4006-8f02-a00b52e741ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:23.011184 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8fc3e-90d0-4006-8f02-a00b52e741ae admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:23.102674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74662c-ed3f-4410-95f1-e2c9f3577477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:23.103708 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74662c-ed3f-4410-95f1-e2c9f3577477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:23.104214 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74662c-ed3f-4410-95f1-e2c9f3577477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:23.104668 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74662c-ed3f-4410-95f1-e2c9f3577477 admin admin] PATH_INFO: `/v3/users/2ca46ba1a656436d9fe208584473d04b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:23.105870 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74662c-ed3f-4410-95f1-e2c9f3577477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca46ba1a656436d9fe208584473d04b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:23.107836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74662c-ed3f-4410-95f1-e2c9f3577477 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:23.114600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:23.236574 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cd74662c-ed3f-4410-95f1-e2c9f3577477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca46ba1a656436d9fe208584473d04b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:23.248701 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cd74662c-ed3f-4410-95f1-e2c9f3577477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca46ba1a656436d9fe208584473d04b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:23.249955 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cd74662c-ed3f-4410-95f1-e2c9f3577477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca46ba1a656436d9fe208584473d04b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:23.283331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cd74662c-ed3f-4410-95f1-e2c9f3577477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca46ba1a656436d9fe208584473d04b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:23.296020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cd74662c-ed3f-4410-95f1-e2c9f3577477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca46ba1a656436d9fe208584473d04b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:23.304906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1443/2863] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:22 2022] DELETE /identity/v3/users/2ca46ba1a656436d9fe208584473d04b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:23.762625 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:23.764036 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:23.786091 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15dfdd-8431-4be1-8702-8a3f2c96a193 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:23.858156 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:23.874511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36aad0-0447-4892-a317-86082a4aff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:23.875172 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36aad0-0447-4892-a317-86082a4aff4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:23.875447 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36aad0-0447-4892-a317-86082a4aff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:23.875715 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36aad0-0447-4892-a317-86082a4aff4e admin admin] PATH_INFO: `/v3/projects/3c7f6821015d4f53aa912c89b5fdeb1a` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:23.876958 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36aad0-0447-4892-a317-86082a4aff4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c7f6821015d4f53aa912c89b5fdeb1a)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:23.879030 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36aad0-0447-4892-a317-86082a4aff4e admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:23.936486 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cb36aad0-0447-4892-a317-86082a4aff4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c7f6821015d4f53aa912c89b5fdeb1a', 'request_id': 'req-cb36aad0-0447-4892-a317-86082a4aff4e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:23.947216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cb36aad0-0447-4892-a317-86082a4aff4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c7f6821015d4f53aa912c89b5fdeb1a', 'request_id': 'req-cb36aad0-0447-4892-a317-86082a4aff4e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:23.959371 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cb36aad0-0447-4892-a317-86082a4aff4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c7f6821015d4f53aa912c89b5fdeb1a', 'request_id': 'req-cb36aad0-0447-4892-a317-86082a4aff4e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:23.976344 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cb36aad0-0447-4892-a317-86082a4aff4e admin admin] The token cache is being invalidate because project 3c7f6821015d4f53aa912c89b5fdeb1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:23.976786 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cb36aad0-0447-4892-a317-86082a4aff4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:23.977733 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-cb36aad0-0447-4892-a317-86082a4aff4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:23.984507 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1421/2864] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:23 2022] DELETE /identity/v3/projects/3c7f6821015d4f53aa912c89b5fdeb1a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:23.992225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826b343-4774-4890-965f-db335d3bb366 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:23.992225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826b343-4774-4890-965f-db335d3bb366 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:23.992225 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826b343-4774-4890-965f-db335d3bb366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:24.001730 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c819495-f8d7-41ab-bf48-a7bcf0c3672b None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:24.002249 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c819495-f8d7-41ab-bf48-a7bcf0c3672b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:24.002600 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c819495-f8d7-41ab-bf48-a7bcf0c3672b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:24.038315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-9c819495-f8d7-41ab-bf48-a7bcf0c3672b None None] MFA Rules not processed for user `9d52bc941fb2491091249353f1c677e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:24.040000 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9c819495-f8d7-41ab-bf48-a7bcf0c3672b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:24.078413 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8826b343-4774-4890-965f-db335d3bb366 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:24.079744 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8826b343-4774-4890-965f-db335d3bb366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:24.131681 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1422/2865] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:24.150930 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c819495-f8d7-41ab-bf48-a7bcf0c3672b None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:24.152353 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-9c819495-f8d7-41ab-bf48-a7bcf0c3672b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:24.153494 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1444/2866] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:24.161754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884c5a1-9676-42a0-95ca-f68f69074ceb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:24.162187 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884c5a1-9676-42a0-95ca-f68f69074ceb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:24.162641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884c5a1-9676-42a0-95ca-f68f69074ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:24.185100 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8884c5a1-9676-42a0-95ca-f68f69074ceb None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:24.186258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8884c5a1-9676-42a0-95ca-f68f69074ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:24.195418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1445/2867] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:24.202498 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884c5a1-9676-42a0-95ca-f68f69074ceb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:24.214011 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884c5a1-9676-42a0-95ca-f68f69074ceb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:24.218862 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309856c5-916a-4706-8f43-5e7d5971950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:24.219472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309856c5-916a-4706-8f43-5e7d5971950b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:24.219741 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309856c5-916a-4706-8f43-5e7d5971950b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:24.220044 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309856c5-916a-4706-8f43-5e7d5971950b admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:24.220685 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309856c5-916a-4706-8f43-5e7d5971950b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:44:24.220980 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309856c5-916a-4706-8f43-5e7d5971950b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:24.222575 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309856c5-916a-4706-8f43-5e7d5971950b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:24.231244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:24.240836 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1446/2868] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:44:24.246787 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309856c5-916a-4706-8f43-5e7d5971950b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:24.259218 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309856c5-916a-4706-8f43-5e7d5971950b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:24.265014 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfae60-bc60-4359-88f7-c4bc564ec952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:24.265627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfae60-bc60-4359-88f7-c4bc564ec952 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:24.265904 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfae60-bc60-4359-88f7-c4bc564ec952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:24.266170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfae60-bc60-4359-88f7-c4bc564ec952 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:24.266808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfae60-bc60-4359-88f7-c4bc564ec952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:24.268754 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfae60-bc60-4359-88f7-c4bc564ec952 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:24.282915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c819495-f8d7-41ab-bf48-a7bcf0c3672b None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:24.292533 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451998b-637d-4259-af02-03d9011e395c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:24.293269 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451998b-637d-4259-af02-03d9011e395c service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:24.293586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451998b-637d-4259-af02-03d9011e395c service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:24.293924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451998b-637d-4259-af02-03d9011e395c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:24.296145 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a451998b-637d-4259-af02-03d9011e395c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:24.302251 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1447/2869] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:24 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:24.306691 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfae60-bc60-4359-88f7-c4bc564ec952 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:24.326643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:24.378668 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451998b-637d-4259-af02-03d9011e395c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:24.380368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451998b-637d-4259-af02-03d9011e395c service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:24.391922 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1423/2870] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:24.397997 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfae60-bc60-4359-88f7-c4bc564ec952 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:24.403882 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7aa0b-99fb-4314-aca8-eb3974183923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:24.404553 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7aa0b-99fb-4314-aca8-eb3974183923 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:24.404837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7aa0b-99fb-4314-aca8-eb3974183923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:24.405110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7aa0b-99fb-4314-aca8-eb3974183923 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:24.406085 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7aa0b-99fb-4314-aca8-eb3974183923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:24.408486 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7aa0b-99fb-4314-aca8-eb3974183923 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:24.438606 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1448/2871] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:24 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:44:24.442926 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451998b-637d-4259-af02-03d9011e395c service nova] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:24.462746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451998b-637d-4259-af02-03d9011e395c service nova] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:24.472661 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0a8a3-8073-4580-976d-9d8fcdcecce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:24.473341 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a8a3-8073-4580-976d-9d8fcdcecce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:24.473658 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a8a3-8073-4580-976d-9d8fcdcecce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:24.473929 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0a8a3-8073-4580-976d-9d8fcdcecce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:24.475689 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0a8a3-8073-4580-976d-9d8fcdcecce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:24.476610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0a8a3-8073-4580-976d-9d8fcdcecce5 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:24.493884 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1424/2872] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:24.495123 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7aa0b-99fb-4314-aca8-eb3974183923 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:24.507053 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7aa0b-99fb-4314-aca8-eb3974183923 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:24.512235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ddd82a-c07c-4ca0-865b-d4d9b027876b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:24.513502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ddd82a-c07c-4ca0-865b-d4d9b027876b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:24.513808 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ddd82a-c07c-4ca0-865b-d4d9b027876b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:24.514252 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ddd82a-c07c-4ca0-865b-d4d9b027876b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:24.515050 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ddd82a-c07c-4ca0-865b-d4d9b027876b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:24.516849 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ddd82a-c07c-4ca0-865b-d4d9b027876b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:24.532235 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1449/2873] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:44:24.536359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0a8a3-8073-4580-976d-9d8fcdcecce5 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:24.552051 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0a8a3-8073-4580-976d-9d8fcdcecce5 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:24.558244 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba089a95-7c6a-4088-a9bc-7827f5db7a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:24.559077 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba089a95-7c6a-4088-a9bc-7827f5db7a5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:24.559457 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba089a95-7c6a-4088-a9bc-7827f5db7a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:24.559852 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba089a95-7c6a-4088-a9bc-7827f5db7a5c admin admin] PATH_INFO: `/v3/projects/4ffc6ebebba5490f84b35ffb827b08d8/users/fd734af142f64b2db23fc59dfde2fb0e/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:24.579246 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba089a95-7c6a-4088-a9bc-7827f5db7a5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ffc6ebebba5490f84b35ffb827b08d8, user_id=fd734af142f64b2db23fc59dfde2fb0e, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:24.583982 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba089a95-7c6a-4088-a9bc-7827f5db7a5c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:24.614190 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1425/2874] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:24 2022] PUT /identity/v3/projects/4ffc6ebebba5490f84b35ffb827b08d8/users/fd734af142f64b2db23fc59dfde2fb0e/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:24.619307 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4c4fe-a738-4d98-bcd4-b71090b9eca3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:24.619597 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4c4fe-a738-4d98-bcd4-b71090b9eca3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:24.619869 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4c4fe-a738-4d98-bcd4-b71090b9eca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:24.658594 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-45e4c4fe-a738-4d98-bcd4-b71090b9eca3 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:24.661600 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-45e4c4fe-a738-4d98-bcd4-b71090b9eca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:24.816201 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1450/2875] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:24.826881 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba089a95-7c6a-4088-a9bc-7827f5db7a5c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:24.828873 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-ba089a95-7c6a-4088-a9bc-7827f5db7a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:24.954232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba089a95-7c6a-4088-a9bc-7827f5db7a5c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:24.958570 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b294f53-c849-4805-bfbf-743507274e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:24.959948 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b294f53-c849-4805-bfbf-743507274e83 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:24.960131 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b294f53-c849-4805-bfbf-743507274e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:24.960443 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b294f53-c849-4805-bfbf-743507274e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:24.971709 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b294f53-c849-4805-bfbf-743507274e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:24.973309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b294f53-c849-4805-bfbf-743507274e83 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:24.986057 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1426/2876] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:26.110557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c8719-7ff5-41fc-8c68-3868721ccf4e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:26.111080 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c8719-7ff5-41fc-8c68-3868721ccf4e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:26.111243 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c8719-7ff5-41fc-8c68-3868721ccf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:26.160593 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5b8c8719-7ff5-41fc-8c68-3868721ccf4e None None] MFA Rules not processed for user `fd734af142f64b2db23fc59dfde2fb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:26.161879 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5b8c8719-7ff5-41fc-8c68-3868721ccf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:26.287932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1451/2877] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:26.303691 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b294f53-c849-4805-bfbf-743507274e83 service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:26.306165 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7b294f53-c849-4805-bfbf-743507274e83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:26.387993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:26.402404 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b294f53-c849-4805-bfbf-743507274e83 service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:26.407162 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038969c3-ea38-4732-b49c-d39b2cdefb0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6160b373ace44ccfaf557b557343bf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:26.408126 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038969c3-ea38-4732-b49c-d39b2cdefb0b service cinder] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:26.408609 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038969c3-ea38-4732-b49c-d39b2cdefb0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:26.409053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038969c3-ea38-4732-b49c-d39b2cdefb0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:26.424897 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038969c3-ea38-4732-b49c-d39b2cdefb0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:26.426359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038969c3-ea38-4732-b49c-d39b2cdefb0b service cinder] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:26.444198 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1427/2878] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:27.999118 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c000749-104d-4028-9d7a-906cc2f2eaf7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:27.999961 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c000749-104d-4028-9d7a-906cc2f2eaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:27.999961 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c000749-104d-4028-9d7a-906cc2f2eaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:28.022546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2c000749-104d-4028-9d7a-906cc2f2eaf7 None None] MFA Rules not processed for user `fd734af142f64b2db23fc59dfde2fb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:28.023820 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2c000749-104d-4028-9d7a-906cc2f2eaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:28.040265 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1452/2879] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:36.434312 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=6 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:36.685147 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038969c3-ea38-4732-b49c-d39b2cdefb0b service cinder] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:36.686615 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-038969c3-ea38-4732-b49c-d39b2cdefb0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:36.847854 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038969c3-ea38-4732-b49c-d39b2cdefb0b service cinder] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:36.854634 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332f322-0255-4380-92e5-68221505341a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6281dcf2927a4ed2870125df8801be78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:36.856309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332f322-0255-4380-92e5-68221505341a service placement] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:36.856309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332f322-0255-4380-92e5-68221505341a service placement] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:36.856309 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332f322-0255-4380-92e5-68221505341a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:36.858192 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-c332f322-0255-4380-92e5-68221505341a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:36.871865 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c332f322-0255-4380-92e5-68221505341a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:36.873504 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c332f322-0255-4380-92e5-68221505341a service placement] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:36.885241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1428/2880] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:38.017020 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:41.828416 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c000749-104d-4028-9d7a-906cc2f2eaf7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:41.830262 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2c000749-104d-4028-9d7a-906cc2f2eaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:41.850382 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c000749-104d-4028-9d7a-906cc2f2eaf7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:41.855558 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf2231-e1c9-49db-a123-3cc30c32cf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:41.856625 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf2231-e1c9-49db-a123-3cc30c32cf61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:41.857050 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf2231-e1c9-49db-a123-3cc30c32cf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:41.857688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf2231-e1c9-49db-a123-3cc30c32cf61 admin admin] PATH_INFO: `/v3/users/e1804a4ef1974e0d8d7d062bd381efc6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:41.859248 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf2231-e1c9-49db-a123-3cc30c32cf61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1804a4ef1974e0d8d7d062bd381efc6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:41.861397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf2231-e1c9-49db-a123-3cc30c32cf61 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:41.924009 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2ecf2231-e1c9-49db-a123-3cc30c32cf61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1804a4ef1974e0d8d7d062bd381efc6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:41.933828 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2ecf2231-e1c9-49db-a123-3cc30c32cf61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1804a4ef1974e0d8d7d062bd381efc6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:41.934907 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2ecf2231-e1c9-49db-a123-3cc30c32cf61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1804a4ef1974e0d8d7d062bd381efc6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:41.953837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2ecf2231-e1c9-49db-a123-3cc30c32cf61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1804a4ef1974e0d8d7d062bd381efc6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:41.962053 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2ecf2231-e1c9-49db-a123-3cc30c32cf61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1804a4ef1974e0d8d7d062bd381efc6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:41.968604 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1453/2881] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:41 2022] DELETE /identity/v3/users/e1804a4ef1974e0d8d7d062bd381efc6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:41.974026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fa42a-7d6e-4968-b593-d2d70be1380b None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:41.976899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fa42a-7d6e-4968-b593-d2d70be1380b None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:41.976899 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fa42a-7d6e-4968-b593-d2d70be1380b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:41.995787 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-3b4fa42a-7d6e-4968-b593-d2d70be1380b None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:41.997054 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3b4fa42a-7d6e-4968-b593-d2d70be1380b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:42.071522 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1429/2882] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:42.081926 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf2231-e1c9-49db-a123-3cc30c32cf61 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:42.083405 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2ecf2231-e1c9-49db-a123-3cc30c32cf61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:42.178335 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf2231-e1c9-49db-a123-3cc30c32cf61 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:42.183175 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f2539-c904-44b0-8a9d-4efc9f601f44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:42.183820 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f2539-c904-44b0-8a9d-4efc9f601f44 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:42.184138 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f2539-c904-44b0-8a9d-4efc9f601f44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:42.185023 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f2539-c904-44b0-8a9d-4efc9f601f44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:42.195990 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f2539-c904-44b0-8a9d-4efc9f601f44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:42.197888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f2539-c904-44b0-8a9d-4efc9f601f44 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:42.211726 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1454/2883] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:44:42.661304 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fa42a-7d6e-4968-b593-d2d70be1380b None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:42.663406 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-3b4fa42a-7d6e-4968-b593-d2d70be1380b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:42.683439 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fa42a-7d6e-4968-b593-d2d70be1380b None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:42.693657 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:42.696120 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:42.696683 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:42.697178 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] PATH_INFO: `/v3/projects/782af202d5ea43838897b844a66093f6` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:42.699368 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=782af202d5ea43838897b844a66093f6)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:42.702587 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:42.784772 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '782af202d5ea43838897b844a66093f6', 'request_id': 'req-4c888e1a-3be2-4575-839b-bfffb0be47e9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:42.793612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '782af202d5ea43838897b844a66093f6', 'request_id': 'req-4c888e1a-3be2-4575-839b-bfffb0be47e9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:42.801545 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '782af202d5ea43838897b844a66093f6', 'request_id': 'req-4c888e1a-3be2-4575-839b-bfffb0be47e9'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:42.802796 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:42.823215 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] The token cache is being invalidate because project 782af202d5ea43838897b844a66093f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:42.824619 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:42.825756 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:42.828434 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1430/2884] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:42 2022] DELETE /identity/v3/projects/782af202d5ea43838897b844a66093f6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:43.051349 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39edbc5-5ec9-48fc-9a23-a4ffd05c3107 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:43.051976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39edbc5-5ec9-48fc-9a23-a4ffd05c3107 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:43.051976 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39edbc5-5ec9-48fc-9a23-a4ffd05c3107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:43.073160 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a39edbc5-5ec9-48fc-9a23-a4ffd05c3107 None None] MFA Rules not processed for user `ed4710f614cc431683a9c2a4f3d9d7b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:43.074511 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a39edbc5-5ec9-48fc-9a23-a4ffd05c3107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:43.137325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:43.150648 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1455/2885] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:44:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:52.824586 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:04.688770 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:04.690723 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:04.752176 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c888e1a-3be2-4575-839b-bfffb0be47e9 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:04.756452 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7349b-98d9-4615-8173-d3ed7302ab68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:04.757075 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7349b-98d9-4615-8173-d3ed7302ab68 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:04.757348 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7349b-98d9-4615-8173-d3ed7302ab68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:04.757633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7349b-98d9-4615-8173-d3ed7302ab68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:04.759808 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-fbc7349b-98d9-4615-8173-d3ed7302ab68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:04.832558 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7349b-98d9-4615-8173-d3ed7302ab68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:04.834280 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7349b-98d9-4615-8173-d3ed7302ab68 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:04.844511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1431/2886] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:14.840281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=50 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:35.578915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe0732-9380-49a8-8b48-4f546d4e8121 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:35.579707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe0732-9380-49a8-8b48-4f546d4e8121 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:35.579707 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe0732-9380-49a8-8b48-4f546d4e8121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:35.603545 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3afe0732-9380-49a8-8b48-4f546d4e8121 None None] MFA Rules not processed for user `fd734af142f64b2db23fc59dfde2fb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:35.604815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3afe0732-9380-49a8-8b48-4f546d4e8121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:35.690032 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1456/2887] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:45:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:35.974641 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7349b-98d9-4615-8173-d3ed7302ab68 service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:35.987037 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7349b-98d9-4615-8173-d3ed7302ab68 service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:35.991304 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba47de-52d7-433a-b812-f9708c34f132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:35.992033 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba47de-52d7-433a-b812-f9708c34f132 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:35.992477 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba47de-52d7-433a-b812-f9708c34f132 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:35.992754 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba47de-52d7-433a-b812-f9708c34f132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:35.994834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b8ba47de-52d7-433a-b812-f9708c34f132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:36.007657 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba47de-52d7-433a-b812-f9708c34f132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:36.009268 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba47de-52d7-433a-b812-f9708c34f132 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:36.022439 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1432/2888] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:45.682398 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=25 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:46.015058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=6 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:10.574887 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe0732-9380-49a8-8b48-4f546d4e8121 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:10.587411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe0732-9380-49a8-8b48-4f546d4e8121 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:10.591915 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa984b33-3dd2-496d-abda-8cc9778947ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:10.592547 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa984b33-3dd2-496d-abda-8cc9778947ef service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:10.592819 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa984b33-3dd2-496d-abda-8cc9778947ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:10.593144 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa984b33-3dd2-496d-abda-8cc9778947ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:10.594898 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fa984b33-3dd2-496d-abda-8cc9778947ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:10.690674 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa984b33-3dd2-496d-abda-8cc9778947ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:10.692203 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa984b33-3dd2-496d-abda-8cc9778947ef service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:10.709697 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1457/2889] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:20.700825 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=34 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:34.269003 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba47de-52d7-433a-b812-f9708c34f132 service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:34.270682 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-b8ba47de-52d7-433a-b812-f9708c34f132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:34.359802 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba47de-52d7-433a-b812-f9708c34f132 service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:34.365585 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deaaffe-491a-4e73-a256-371ec425de2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:34.366305 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaaffe-491a-4e73-a256-371ec425de2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:34.366633 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaaffe-491a-4e73-a256-371ec425de2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:34.366950 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaaffe-491a-4e73-a256-371ec425de2e admin admin] PATH_INFO: `/v3/users/fd734af142f64b2db23fc59dfde2fb0e` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:34.368571 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deaaffe-491a-4e73-a256-371ec425de2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd734af142f64b2db23fc59dfde2fb0e)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:34.370924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deaaffe-491a-4e73-a256-371ec425de2e admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:34.433973 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7deaaffe-491a-4e73-a256-371ec425de2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd734af142f64b2db23fc59dfde2fb0e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.450110 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7deaaffe-491a-4e73-a256-371ec425de2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd734af142f64b2db23fc59dfde2fb0e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.457970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7deaaffe-491a-4e73-a256-371ec425de2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd734af142f64b2db23fc59dfde2fb0e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.467413 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7deaaffe-491a-4e73-a256-371ec425de2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd734af142f64b2db23fc59dfde2fb0e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.468577 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-7deaaffe-491a-4e73-a256-371ec425de2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd734af142f64b2db23fc59dfde2fb0e'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.479241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1433/2890] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:34 2022] DELETE /identity/v3/users/fd734af142f64b2db23fc59dfde2fb0e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:34.701445 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa984b33-3dd2-496d-abda-8cc9778947ef service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:34.717291 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa984b33-3dd2-496d-abda-8cc9778947ef service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:34.800060 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035e15d-6d26-4cc2-929a-3e49c500902b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:34.800927 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035e15d-6d26-4cc2-929a-3e49c500902b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:34.801395 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035e15d-6d26-4cc2-929a-3e49c500902b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:34.801850 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035e15d-6d26-4cc2-929a-3e49c500902b admin admin] PATH_INFO: `/v3/projects/4ffc6ebebba5490f84b35ffb827b08d8` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:34.803357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035e15d-6d26-4cc2-929a-3e49c500902b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ffc6ebebba5490f84b35ffb827b08d8)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:34.805824 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035e15d-6d26-4cc2-929a-3e49c500902b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:34.862689 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4035e15d-6d26-4cc2-929a-3e49c500902b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ffc6ebebba5490f84b35ffb827b08d8', 'request_id': 'req-4035e15d-6d26-4cc2-929a-3e49c500902b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.875540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4035e15d-6d26-4cc2-929a-3e49c500902b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ffc6ebebba5490f84b35ffb827b08d8', 'request_id': 'req-4035e15d-6d26-4cc2-929a-3e49c500902b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.886663 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4035e15d-6d26-4cc2-929a-3e49c500902b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ffc6ebebba5490f84b35ffb827b08d8', 'request_id': 'req-4035e15d-6d26-4cc2-929a-3e49c500902b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.902681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4035e15d-6d26-4cc2-929a-3e49c500902b admin admin] The token cache is being invalidate because project 4ffc6ebebba5490f84b35ffb827b08d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:34.902986 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4035e15d-6d26-4cc2-929a-3e49c500902b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.903643 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4035e15d-6d26-4cc2-929a-3e49c500902b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.905378 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1458/2891] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:34 2022] DELETE /identity/v3/projects/4ffc6ebebba5490f84b35ffb827b08d8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:34.913123 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9178c7-89ff-460c-904f-51e55d1881c6 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:34.913612 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9178c7-89ff-460c-904f-51e55d1881c6 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:34.913915 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9178c7-89ff-460c-904f-51e55d1881c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:34.943564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-4f9178c7-89ff-460c-904f-51e55d1881c6 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:34.944999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-4f9178c7-89ff-460c-904f-51e55d1881c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:35.025047 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1434/2892] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:35.033564 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412969c-5996-4aa8-9122-3c20d6e8c7a0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.033868 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412969c-5996-4aa8-9122-3c20d6e8c7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.034165 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412969c-5996-4aa8-9122-3c20d6e8c7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.057272 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b412969c-5996-4aa8-9122-3c20d6e8c7a0 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:35.058581 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b412969c-5996-4aa8-9122-3c20d6e8c7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:35.070332 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1459/2893] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:35.075169 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9178c7-89ff-460c-904f-51e55d1881c6 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.085722 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9178c7-89ff-460c-904f-51e55d1881c6 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.091009 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b9670-334f-4717-a935-8d89217edb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.091649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b9670-334f-4717-a935-8d89217edb9b admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.091958 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b9670-334f-4717-a935-8d89217edb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.092233 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b9670-334f-4717-a935-8d89217edb9b admin admin] PATH_INFO: `/v3/domains` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.092900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b9670-334f-4717-a935-8d89217edb9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:35.093199 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b9670-334f-4717-a935-8d89217edb9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.094834 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b9670-334f-4717-a935-8d89217edb9b admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.109357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1435/2894] 10.222.0.62 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:35.115608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412969c-5996-4aa8-9122-3c20d6e8c7a0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.137687 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412969c-5996-4aa8-9122-3c20d6e8c7a0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.146113 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f517ae7-3492-4aa5-ac57-469788df5a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.147116 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f517ae7-3492-4aa5-ac57-469788df5a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.147538 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f517ae7-3492-4aa5-ac57-469788df5a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.148450 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f517ae7-3492-4aa5-ac57-469788df5a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.149540 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f517ae7-3492-4aa5-ac57-469788df5a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.153157 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f517ae7-3492-4aa5-ac57-469788df5a5e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.203324 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1460/2895] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:35 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:35.208669 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b9670-334f-4717-a935-8d89217edb9b admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.219069 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b9670-334f-4717-a935-8d89217edb9b admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.223198 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:35.299395 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a804f-25c8-404e-a0c1-2d3db7cf3e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.300342 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a804f-25c8-404e-a0c1-2d3db7cf3e76 admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.300509 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a804f-25c8-404e-a0c1-2d3db7cf3e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.300795 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a804f-25c8-404e-a0c1-2d3db7cf3e76 admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.301455 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a804f-25c8-404e-a0c1-2d3db7cf3e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.303502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a804f-25c8-404e-a0c1-2d3db7cf3e76 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.332334 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1436/2896] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:35 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:35.337957 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f517ae7-3492-4aa5-ac57-469788df5a5e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.354106 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f517ae7-3492-4aa5-ac57-469788df5a5e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.359467 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d19af-b81d-4505-91b7-14e918dc4357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.360140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d19af-b81d-4505-91b7-14e918dc4357 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.360411 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d19af-b81d-4505-91b7-14e918dc4357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.360679 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d19af-b81d-4505-91b7-14e918dc4357 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.361362 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d19af-b81d-4505-91b7-14e918dc4357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.363021 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d19af-b81d-4505-91b7-14e918dc4357 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.375421 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1461/2897] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:35.380889 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a804f-25c8-404e-a0c1-2d3db7cf3e76 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.394493 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a804f-25c8-404e-a0c1-2d3db7cf3e76 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.401423 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b5d51-b654-4c02-8441-fde27c3ee88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.402058 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b5d51-b654-4c02-8441-fde27c3ee88d admin admin] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.402339 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b5d51-b654-4c02-8441-fde27c3ee88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.402817 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b5d51-b654-4c02-8441-fde27c3ee88d admin admin] PATH_INFO: `/v3/roles` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.404082 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b5d51-b654-4c02-8441-fde27c3ee88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.407204 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b5d51-b654-4c02-8441-fde27c3ee88d admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.424666 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1437/2898] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:35.431321 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d19af-b81d-4505-91b7-14e918dc4357 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.449030 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d19af-b81d-4505-91b7-14e918dc4357 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.456526 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b787a8a-d4c5-40b4-a917-12a9ca18d8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.457641 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b787a8a-d4c5-40b4-a917-12a9ca18d8c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.458088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b787a8a-d4c5-40b4-a917-12a9ca18d8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.458543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b787a8a-d4c5-40b4-a917-12a9ca18d8c9 admin admin] PATH_INFO: `/v3/projects/3b373831f3c5412eb2170fef6580c110/users/d762a5837f034fc2aafa0efa33708530/roles/3100f6054f7d4d7b984fe19a4d8ba291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.484571 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b787a8a-d4c5-40b4-a917-12a9ca18d8c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b373831f3c5412eb2170fef6580c110, user_id=d762a5837f034fc2aafa0efa33708530, role_id=3100f6054f7d4d7b984fe19a4d8ba291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.489339 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b787a8a-d4c5-40b4-a917-12a9ca18d8c9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.513803 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1462/2899] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:35 2022] PUT /identity/v3/projects/3b373831f3c5412eb2170fef6580c110/users/d762a5837f034fc2aafa0efa33708530/roles/3100f6054f7d4d7b984fe19a4d8ba291 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:35.520905 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d75591-85b8-4ce1-b9ab-074ef10efaf7 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.521432 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d75591-85b8-4ce1-b9ab-074ef10efaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.521919 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d75591-85b8-4ce1-b9ab-074ef10efaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.557132 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-36d75591-85b8-4ce1-b9ab-074ef10efaf7 None None] MFA Rules not processed for user `82b69c02f3fd4ddc81686f03143f0651`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:35.559515 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-36d75591-85b8-4ce1-b9ab-074ef10efaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:35.644821 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1438/2900] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:35.655781 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b787a8a-d4c5-40b4-a917-12a9ca18d8c9 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.657992 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1b787a8a-d4c5-40b4-a917-12a9ca18d8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:35.683357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:35.733981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b787a8a-d4c5-40b4-a917-12a9ca18d8c9 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.738199 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c8e8-3597-4083-a84d-a9a91ef99853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.738796 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c8e8-3597-4083-a84d-a9a91ef99853 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.739090 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c8e8-3597-4083-a84d-a9a91ef99853 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.739354 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c8e8-3597-4083-a84d-a9a91ef99853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.749658 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0c8e8-3597-4083-a84d-a9a91ef99853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.751317 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0c8e8-3597-4083-a84d-a9a91ef99853 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.764682 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1463/2901] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:46:40.828778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25978c6d-5ec5-4a0c-936f-d0e02d2531f3 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:40.829304 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25978c6d-5ec5-4a0c-936f-d0e02d2531f3 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:40.829701 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25978c6d-5ec5-4a0c-936f-d0e02d2531f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:40.875610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-25978c6d-5ec5-4a0c-936f-d0e02d2531f3 None None] MFA Rules not processed for user `d762a5837f034fc2aafa0efa33708530`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:40.876924 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-25978c6d-5ec5-4a0c-936f-d0e02d2531f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:41.004601 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1439/2902] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:46:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:41.010491 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c8e8-3597-4083-a84d-a9a91ef99853 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:41.024530 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c8e8-3597-4083-a84d-a9a91ef99853 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:41.030925 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacad2e-6cf6-4f75-8d2d-9455af4fc9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:41.031800 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacad2e-6cf6-4f75-8d2d-9455af4fc9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:41.032188 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacad2e-6cf6-4f75-8d2d-9455af4fc9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:41.032546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacad2e-6cf6-4f75-8d2d-9455af4fc9b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:41.033316 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacad2e-6cf6-4f75-8d2d-9455af4fc9b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:41.036406 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacad2e-6cf6-4f75-8d2d-9455af4fc9b4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:41.097949 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1464/2903] 10.222.0.62 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:41 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:41.106124 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25978c6d-5ec5-4a0c-936f-d0e02d2531f3 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:41.119676 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25978c6d-5ec5-4a0c-936f-d0e02d2531f3 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:41.138351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:41.216163 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37504acd-db29-40b4-b548-6b54a3e92b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:41.216985 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37504acd-db29-40b4-b548-6b54a3e92b4b admin admin] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:41.217953 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37504acd-db29-40b4-b548-6b54a3e92b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:41.218167 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37504acd-db29-40b4-b548-6b54a3e92b4b admin admin] PATH_INFO: `/v3/users` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:41.218964 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37504acd-db29-40b4-b548-6b54a3e92b4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:41.221745 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37504acd-db29-40b4-b548-6b54a3e92b4b admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:41.257684 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1440/2904] 10.222.0.62 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:41 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:41.263024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacad2e-6cf6-4f75-8d2d-9455af4fc9b4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:41.277942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacad2e-6cf6-4f75-8d2d-9455af4fc9b4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:41.283126 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084362d-3ce0-4ddf-aea4-85b874df0fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:41.283802 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084362d-3ce0-4ddf-aea4-85b874df0fcf admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:41.284109 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084362d-3ce0-4ddf-aea4-85b874df0fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:41.284468 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084362d-3ce0-4ddf-aea4-85b874df0fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:41.285324 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084362d-3ce0-4ddf-aea4-85b874df0fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:41.287472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084362d-3ce0-4ddf-aea4-85b874df0fcf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:41.303607 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1465/2905] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:41.308685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37504acd-db29-40b4-b548-6b54a3e92b4b admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:41.320843 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37504acd-db29-40b4-b548-6b54a3e92b4b admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:41.328040 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f7e21-032a-41e1-a3a2-077a3c26130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:41.328908 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f7e21-032a-41e1-a3a2-077a3c26130c admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:41.329266 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f7e21-032a-41e1-a3a2-077a3c26130c admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:41.329631 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f7e21-032a-41e1-a3a2-077a3c26130c admin admin] PATH_INFO: `/v3/projects/38f2b39b83804fcaaff94680c12449c2/users/f2bbf90b05c146a4a45ce2ad9adcf508/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:41.347583 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f7e21-032a-41e1-a3a2-077a3c26130c admin admin] RBAC: Authorizing `identity:create_grant(project_id=38f2b39b83804fcaaff94680c12449c2, user_id=f2bbf90b05c146a4a45ce2ad9adcf508, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:41.350451 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f7e21-032a-41e1-a3a2-077a3c26130c admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:41.376431 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1441/2906] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:41 2022] PUT /identity/v3/projects/38f2b39b83804fcaaff94680c12449c2/users/f2bbf90b05c146a4a45ce2ad9adcf508/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:41.380984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084362d-3ce0-4ddf-aea4-85b874df0fcf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:41.397263 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084362d-3ce0-4ddf-aea4-85b874df0fcf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:41.466181 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdf121-aa39-421e-a044-331c1eb92cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:41.466946 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdf121-aa39-421e-a044-331c1eb92cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:41.467222 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdf121-aa39-421e-a044-331c1eb92cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:41.467548 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdf121-aa39-421e-a044-331c1eb92cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:41.468249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cdf121-aa39-421e-a044-331c1eb92cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:41.469996 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cdf121-aa39-421e-a044-331c1eb92cb8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:41.488846 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1466/2907] 10.222.0.62 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:41.490502 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f7e21-032a-41e1-a3a2-077a3c26130c admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:41.503106 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f7e21-032a-41e1-a3a2-077a3c26130c admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:41.510569 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a37630-d007-41f8-ade6-16743d22185f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:41.511694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a37630-d007-41f8-ade6-16743d22185f admin admin] REQUEST_METHOD: `PUT` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:41.512253 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a37630-d007-41f8-ade6-16743d22185f admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:41.512741 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a37630-d007-41f8-ade6-16743d22185f admin admin] PATH_INFO: `/v3/projects/38f2b39b83804fcaaff94680c12449c2/users/f2bbf90b05c146a4a45ce2ad9adcf508/roles/ae7ca58f44a54413bcd33ee40caf4fe1` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:41.515536 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a37630-d007-41f8-ade6-16743d22185f admin admin] RBAC: Authorizing `identity:create_grant(project_id=38f2b39b83804fcaaff94680c12449c2, user_id=f2bbf90b05c146a4a45ce2ad9adcf508, role_id=ae7ca58f44a54413bcd33ee40caf4fe1)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:41.518967 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a37630-d007-41f8-ade6-16743d22185f admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:41.531591 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1442/2908] 10.222.0.62 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:41 2022] PUT /identity/v3/projects/38f2b39b83804fcaaff94680c12449c2/users/f2bbf90b05c146a4a45ce2ad9adcf508/roles/ae7ca58f44a54413bcd33ee40caf4fe1 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:47.263735 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7b725-10f0-41ca-8e44-ca6572d3272f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:47.264637 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7b725-10f0-41ca-8e44-ca6572d3272f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:47.264637 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7b725-10f0-41ca-8e44-ca6572d3272f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:47.316732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b4c7b725-10f0-41ca-8e44-ca6572d3272f None None] MFA Rules not processed for user `f2bbf90b05c146a4a45ce2ad9adcf508`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:47.318981 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b4c7b725-10f0-41ca-8e44-ca6572d3272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:47.459688 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1467/2909] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:46:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:47.481120 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a37630-d007-41f8-ade6-16743d22185f admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:47.485270 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-77a37630-d007-41f8-ade6-16743d22185f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:47.596461 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a37630-d007-41f8-ade6-16743d22185f admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:47.603242 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d43f74-2b16-4b1b-baef-d6bf9dc40b12 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfe36a14a11493397e4018b773d3cc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:47.604357 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d43f74-2b16-4b1b-baef-d6bf9dc40b12 service glance] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:47.604743 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d43f74-2b16-4b1b-baef-d6bf9dc40b12 service glance] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:47.605182 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d43f74-2b16-4b1b-baef-d6bf9dc40b12 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:47.710259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d43f74-2b16-4b1b-baef-d6bf9dc40b12 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:47.711705 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d43f74-2b16-4b1b-baef-d6bf9dc40b12 service glance] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:47.727276 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1443/2910] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:47.882670 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf6f12-8e49-485d-b428-726bb070e2d7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:47.883192 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf6f12-8e49-485d-b428-726bb070e2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:47.883640 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf6f12-8e49-485d-b428-726bb070e2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:47.906164 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:47.984557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-eedf6f12-8e49-485d-b428-726bb070e2d7 None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:47.986091 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eedf6f12-8e49-485d-b428-726bb070e2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:48.072920 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1468/2911] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:46:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:48.138241 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d43f74-2b16-4b1b-baef-d6bf9dc40b12 service glance] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:48.139746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-81d43f74-2b16-4b1b-baef-d6bf9dc40b12 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:48.181490 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:48.261782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d43f74-2b16-4b1b-baef-d6bf9dc40b12 service glance] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:48.267883 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b1ad3-6f62-4cad-8688-11a7f648ecef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:48.269095 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b1ad3-6f62-4cad-8688-11a7f648ecef service swift] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:48.269518 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b1ad3-6f62-4cad-8688-11a7f648ecef service swift] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:48.269714 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b1ad3-6f62-4cad-8688-11a7f648ecef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:48.281997 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b1ad3-6f62-4cad-8688-11a7f648ecef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:48.283610 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b1ad3-6f62-4cad-8688-11a7f648ecef service swift] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:48.297813 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1444/2912] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:46:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:46:58.069108 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=40 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:58.292945 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=31 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:58.598811 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7e88f-7dc0-4f68-b0fe-084fa401c45b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:58.599127 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7e88f-7dc0-4f68-b0fe-084fa401c45b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:58.599453 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7e88f-7dc0-4f68-b0fe-084fa401c45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:58.622690 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cfe7e88f-7dc0-4f68-b0fe-084fa401c45b None None] MFA Rules not processed for user `d762a5837f034fc2aafa0efa33708530`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:58.624250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cfe7e88f-7dc0-4f68-b0fe-084fa401c45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:58.633592 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1469/2913] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:46:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:58.680447 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b1ad3-6f62-4cad-8688-11a7f648ecef service swift] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:58.691091 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b1ad3-6f62-4cad-8688-11a7f648ecef service swift] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:58.695639 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a7150-6e51-4723-beb4-a88a9ea0b2dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cfe36a14a11493397e4018b773d3cc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:58.696294 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a7150-6e51-4723-beb4-a88a9ea0b2dc service glance] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:58.696571 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a7150-6e51-4723-beb4-a88a9ea0b2dc service glance] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:58.696839 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a7150-6e51-4723-beb4-a88a9ea0b2dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:58.699053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-687a7150-6e51-4723-beb4-a88a9ea0b2dc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:58.785942 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a7150-6e51-4723-beb4-a88a9ea0b2dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:58.787422 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a7150-6e51-4723-beb4-a88a9ea0b2dc service glance] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:58.799259 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1445/2914] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:59.416685 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7e88f-7dc0-4f68-b0fe-084fa401c45b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:59.418611 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cfe7e88f-7dc0-4f68-b0fe-084fa401c45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:59.525162 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7e88f-7dc0-4f68-b0fe-084fa401c45b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:59.530732 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95a7eb-0e6d-45d6-8ca6-0f0e8fd273ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6281dcf2927a4ed2870125df8801be78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:59.531345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95a7eb-0e6d-45d6-8ca6-0f0e8fd273ed service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:59.531624 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95a7eb-0e6d-45d6-8ca6-0f0e8fd273ed service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:59.531906 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95a7eb-0e6d-45d6-8ca6-0f0e8fd273ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:59.533837 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1c95a7eb-0e6d-45d6-8ca6-0f0e8fd273ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:59.547258 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95a7eb-0e6d-45d6-8ca6-0f0e8fd273ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:59.548912 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95a7eb-0e6d-45d6-8ca6-0f0e8fd273ed service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:59.560277 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1470/2915] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:47:00.975336 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d605247-91c8-4915-89c0-677845d38a39 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:00.975975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d605247-91c8-4915-89c0-677845d38a39 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:00.975975 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d605247-91c8-4915-89c0-677845d38a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:00.998743 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-6d605247-91c8-4915-89c0-677845d38a39 None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:01.000435 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-6d605247-91c8-4915-89c0-677845d38a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:01.009426 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1446/2916] 10.222.0.62 () {58 vars in 1009 bytes} [Thu Sep 1 16:47:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:47:01.023017 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95a7eb-0e6d-45d6-8ca6-0f0e8fd273ed service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:01.035875 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95a7eb-0e6d-45d6-8ca6-0f0e8fd273ed service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:01.041809 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d447fc7-95ef-4c27-bb0f-0d7dc0855a3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:01.042469 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d447fc7-95ef-4c27-bb0f-0d7dc0855a3d service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:01.042815 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d447fc7-95ef-4c27-bb0f-0d7dc0855a3d service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:01.043091 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d447fc7-95ef-4c27-bb0f-0d7dc0855a3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:01.064458 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d447fc7-95ef-4c27-bb0f-0d7dc0855a3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:01.066250 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d447fc7-95ef-4c27-bb0f-0d7dc0855a3d service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:01.077677 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1471/2917] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:47:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:47:08.925390 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d605247-91c8-4915-89c0-677845d38a39 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:08.987752 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d605247-91c8-4915-89c0-677845d38a39 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:08.992074 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5cf40-ec29-4088-ab25-88af0b188158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:08.992724 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5cf40-ec29-4088-ab25-88af0b188158 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:08.992999 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5cf40-ec29-4088-ab25-88af0b188158 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:08.993264 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5cf40-ec29-4088-ab25-88af0b188158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:08.995098 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e5e5cf40-ec29-4088-ab25-88af0b188158 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:09.005824 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5cf40-ec29-4088-ab25-88af0b188158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:09.007511 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5cf40-ec29-4088-ab25-88af0b188158 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:09.017207 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1447/2918] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:47:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:47:11.074003 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=41 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:13.467175 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58df94-cca3-489f-a2e5-9909de84502f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:13.467920 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58df94-cca3-489f-a2e5-9909de84502f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:13.467920 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58df94-cca3-489f-a2e5-9909de84502f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:13.491116 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4a58df94-cca3-489f-a2e5-9909de84502f None None] MFA Rules not processed for user `ed4710f614cc431683a9c2a4f3d9d7b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:13.492397 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4a58df94-cca3-489f-a2e5-9909de84502f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:13.561725 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1472/2919] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:47:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:47:17.300559 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5cf40-ec29-4088-ab25-88af0b188158 service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:17.302140 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-e5e5cf40-ec29-4088-ab25-88af0b188158 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:17.322970 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5cf40-ec29-4088-ab25-88af0b188158 service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:17.331175 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3d921-2417-47c5-990c-8fde9e0217b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:17.332499 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3d921-2417-47c5-990c-8fde9e0217b4 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:17.333104 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3d921-2417-47c5-990c-8fde9e0217b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:17.334026 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3d921-2417-47c5-990c-8fde9e0217b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:17.337015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-2db3d921-2417-47c5-990c-8fde9e0217b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:17.360850 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3d921-2417-47c5-990c-8fde9e0217b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:17.363918 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3d921-2417-47c5-990c-8fde9e0217b4 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:17.385265 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1448/2920] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:47:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:47:23.554730 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=25 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:27.377928 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=69 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:29.205331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58df94-cca3-489f-a2e5-9909de84502f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:29.216772 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58df94-cca3-489f-a2e5-9909de84502f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:29.222544 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2ce8f-aaeb-441d-b607-d09655af0118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:29.223422 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2ce8f-aaeb-441d-b607-d09655af0118 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:29.223762 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2ce8f-aaeb-441d-b607-d09655af0118 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:29.224170 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2ce8f-aaeb-441d-b607-d09655af0118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:29.235474 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2ce8f-aaeb-441d-b607-d09655af0118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:29.237098 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2ce8f-aaeb-441d-b607-d09655af0118 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:29.249466 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1473/2921] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:47:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:47:31.188300 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3d921-2417-47c5-990c-8fde9e0217b4 service nova] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:31.203232 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3d921-2417-47c5-990c-8fde9e0217b4 service nova] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:31.211429 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486a43a-bc90-473d-a990-f4a95067b899 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:31.212145 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486a43a-bc90-473d-a990-f4a95067b899 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:31.212419 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486a43a-bc90-473d-a990-f4a95067b899 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:31.212689 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486a43a-bc90-473d-a990-f4a95067b899 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:31.214905 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-a486a43a-bc90-473d-a990-f4a95067b899 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:31.229921 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486a43a-bc90-473d-a990-f4a95067b899 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:31.231721 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486a43a-bc90-473d-a990-f4a95067b899 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:31.244763 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1449/2922] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:47:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:47:39.244563 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:41.239024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=6 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:47.043491 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2ce8f-aaeb-441d-b607-d09655af0118 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:47.076872 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2ce8f-aaeb-441d-b607-d09655af0118 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:47.087195 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f63c2f-07fe-4281-a6d7-ba04380d0dd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:47.089669 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63c2f-07fe-4281-a6d7-ba04380d0dd3 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:47.090873 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63c2f-07fe-4281-a6d7-ba04380d0dd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:47.091455 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63c2f-07fe-4281-a6d7-ba04380d0dd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:47.095440 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b1f63c2f-07fe-4281-a6d7-ba04380d0dd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:47.121527 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f63c2f-07fe-4281-a6d7-ba04380d0dd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:47.125267 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f63c2f-07fe-4281-a6d7-ba04380d0dd3 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:47.148260 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1474/2923] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:48:55.366982 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb9282-eb25-4c8d-a839-d5b6fc64cb03 None None] REQUEST_METHOD: `POST` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:55.367838 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb9282-eb25-4c8d-a839-d5b6fc64cb03 None None] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:55.368281 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb9282-eb25-4c8d-a839-d5b6fc64cb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:55.394682 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.auth.core [None req-38fb9282-eb25-4c8d-a839-d5b6fc64cb03 None None] MFA Rules not processed for user `24705faaf5054802a556749c7ba82ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:48:55.396228 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-38fb9282-eb25-4c8d-a839-d5b6fc64cb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:55.407826 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1450/2924] 10.222.0.62 () {58 vars in 967 bytes} [Thu Sep 1 16:48:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:48:55.483561 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f63c2f-07fe-4281-a6d7-ba04380d0dd3 service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:55.503962 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f63c2f-07fe-4281-a6d7-ba04380d0dd3 service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:55.513054 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e99565-8c4f-468c-b7b8-f76a11548b49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56f9bc7262b64bbab0f670576c6bdfdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:55.514086 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e99565-8c4f-468c-b7b8-f76a11548b49 service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:55.514557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e99565-8c4f-468c-b7b8-f76a11548b49 service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:55.515016 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e99565-8c4f-468c-b7b8-f76a11548b49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:55.531117 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e99565-8c4f-468c-b7b8-f76a11548b49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:55.534216 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e99565-8c4f-468c-b7b8-f76a11548b49 service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:55.547763 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1475/2925] 10.222.0.62 () {60 vars in 1384 bytes} [Thu Sep 1 16:48:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:49:00.386297 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb9282-eb25-4c8d-a839-d5b6fc64cb03 None None] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:49:00.405762 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb9282-eb25-4c8d-a839-d5b6fc64cb03 None None] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:49:00.411424 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee714d-660d-4d0c-a16c-6ad4675b18cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6281dcf2927a4ed2870125df8801be78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:49:00.412183 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee714d-660d-4d0c-a16c-6ad4675b18cb service placement] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:49:00.412546 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee714d-660d-4d0c-a16c-6ad4675b18cb service placement] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:49:00.413009 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee714d-660d-4d0c-a16c-6ad4675b18cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:49:00.415003 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-47ee714d-660d-4d0c-a16c-6ad4675b18cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:49:00.433630 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee714d-660d-4d0c-a16c-6ad4675b18cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:49:00.436227 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee714d-660d-4d0c-a16c-6ad4675b18cb service placement] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:49:00.452563 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1451/2926] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:49:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:49:04.857546 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e99565-8c4f-468c-b7b8-f76a11548b49 service swift] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:49:04.964320 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e99565-8c4f-468c-b7b8-f76a11548b49 service swift] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:49:04.971249 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66139f4e-bc82-4f5b-b0db-fd9c2a99d2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:49:04.972390 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66139f4e-bc82-4f5b-b0db-fd9c2a99d2d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:49:04.974551 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66139f4e-bc82-4f5b-b0db-fd9c2a99d2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:49:04.974978 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66139f4e-bc82-4f5b-b0db-fd9c2a99d2d1 admin admin] PATH_INFO: `/v3/users/d762a5837f034fc2aafa0efa33708530` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:49:04.976807 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66139f4e-bc82-4f5b-b0db-fd9c2a99d2d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d762a5837f034fc2aafa0efa33708530)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:49:04.980429 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66139f4e-bc82-4f5b-b0db-fd9c2a99d2d1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:49:05.072150 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-66139f4e-bc82-4f5b-b0db-fd9c2a99d2d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd762a5837f034fc2aafa0efa33708530'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.086049 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-66139f4e-bc82-4f5b-b0db-fd9c2a99d2d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd762a5837f034fc2aafa0efa33708530'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.086957 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-66139f4e-bc82-4f5b-b0db-fd9c2a99d2d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd762a5837f034fc2aafa0efa33708530'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.106321 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-66139f4e-bc82-4f5b-b0db-fd9c2a99d2d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd762a5837f034fc2aafa0efa33708530'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.115524 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-66139f4e-bc82-4f5b-b0db-fd9c2a99d2d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd762a5837f034fc2aafa0efa33708530'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.122982 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1476/2927] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:49:04 2022] DELETE /identity/v3/users/d762a5837f034fc2aafa0efa33708530 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:49:05.371638 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee714d-660d-4d0c-a16c-6ad4675b18cb service placement] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:49:05.385603 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee714d-660d-4d0c-a16c-6ad4675b18cb service placement] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:49:05.490173 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:49:05.490847 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:49:05.491216 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:49:05.491584 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] PATH_INFO: `/v3/projects/3b373831f3c5412eb2170fef6580c110` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:49:05.493992 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b373831f3c5412eb2170fef6580c110)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:49:05.496632 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:49:05.575642 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b373831f3c5412eb2170fef6580c110', 'request_id': 'req-64ec7b0f-bd3f-44fa-b894-458d000007fa'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.587550 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b373831f3c5412eb2170fef6580c110', 'request_id': 'req-64ec7b0f-bd3f-44fa-b894-458d000007fa'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.598356 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b373831f3c5412eb2170fef6580c110', 'request_id': 'req-64ec7b0f-bd3f-44fa-b894-458d000007fa'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.612150 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] The token cache is being invalidate because project 3b373831f3c5412eb2170fef6580c110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:49:05.612447 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.613228 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.614856 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1452/2928] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:49:05 2022] DELETE /identity/v3/projects/3b373831f3c5412eb2170fef6580c110 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:49:05.620140 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66139f4e-bc82-4f5b-b0db-fd9c2a99d2d1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:49:05.621675 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-66139f4e-bc82-4f5b-b0db-fd9c2a99d2d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:49:05.639022 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66139f4e-bc82-4f5b-b0db-fd9c2a99d2d1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:49:05.706319 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:49:05.719789 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceda86b-8ab4-48fd-ad7a-ea0429e08896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:49:05.720565 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceda86b-8ab4-48fd-ad7a-ea0429e08896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:49:05.720884 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceda86b-8ab4-48fd-ad7a-ea0429e08896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:49:05.721171 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceda86b-8ab4-48fd-ad7a-ea0429e08896 admin admin] PATH_INFO: `/v3/users/f2bbf90b05c146a4a45ce2ad9adcf508` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:49:05.722737 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceda86b-8ab4-48fd-ad7a-ea0429e08896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2bbf90b05c146a4a45ce2ad9adcf508)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:49:05.725321 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceda86b-8ab4-48fd-ad7a-ea0429e08896 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:49:05.789969 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ceda86b-8ab4-48fd-ad7a-ea0429e08896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2bbf90b05c146a4a45ce2ad9adcf508'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.800293 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ceda86b-8ab4-48fd-ad7a-ea0429e08896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2bbf90b05c146a4a45ce2ad9adcf508'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.801255 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ceda86b-8ab4-48fd-ad7a-ea0429e08896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2bbf90b05c146a4a45ce2ad9adcf508'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.820608 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ceda86b-8ab4-48fd-ad7a-ea0429e08896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2bbf90b05c146a4a45ce2ad9adcf508'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.834584 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ceda86b-8ab4-48fd-ad7a-ea0429e08896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2bbf90b05c146a4a45ce2ad9adcf508'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:05.846111 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1477/2929] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:49:05 2022] DELETE /identity/v3/users/f2bbf90b05c146a4a45ce2ad9adcf508 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:49:06.074892 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:49:06.077366 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:49:06.098010 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec7b0f-bd3f-44fa-b894-458d000007fa admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:49:06.187234 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:49:06.188408 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:49:06.188887 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:49:06.189359 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] PATH_INFO: `/v3/projects/38f2b39b83804fcaaff94680c12449c2` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:49:06.191326 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38f2b39b83804fcaaff94680c12449c2)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:49:06.193900 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:49:06.210920 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:49:06.267694 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38f2b39b83804fcaaff94680c12449c2', 'request_id': 'req-841af6cd-15d0-402f-98a4-48637ed1c8f6'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:06.278109 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38f2b39b83804fcaaff94680c12449c2', 'request_id': 'req-841af6cd-15d0-402f-98a4-48637ed1c8f6'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:06.288685 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38f2b39b83804fcaaff94680c12449c2', 'request_id': 'req-841af6cd-15d0-402f-98a4-48637ed1c8f6'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:06.305028 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] The token cache is being invalidate because project 38f2b39b83804fcaaff94680c12449c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:49:06.306072 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:06.307387 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:49:06.310624 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1453/2930] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:49:06 2022] DELETE /identity/v3/projects/38f2b39b83804fcaaff94680c12449c2 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:49:14.458557 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e99b80-cfb1-40df-99cc-c1e086435e16 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:49:14.459357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e99b80-cfb1-40df-99cc-c1e086435e16 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:49:14.459357 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e99b80-cfb1-40df-99cc-c1e086435e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:49:14.488885 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c8e99b80-cfb1-40df-99cc-c1e086435e16 None None] MFA Rules not processed for user `ed4710f614cc431683a9c2a4f3d9d7b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:49:14.490833 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c8e99b80-cfb1-40df-99cc-c1e086435e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:49:14.581019 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1478/2931] 10.222.0.62 () {58 vars in 958 bytes} [Thu Sep 1 16:49:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:49:16.299692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:49:24.569133 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=54,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:49:25.358315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:49:25.359780 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:49:25.452406 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841af6cd-15d0-402f-98a4-48637ed1c8f6 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:49:25.459154 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ce8a7-b2f8-402c-8124-3a978cb9a554 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2912c02ff6db4cbc97c80cd923238136', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:49:25.459769 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ce8a7-b2f8-402c-8124-3a978cb9a554 service nova] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:49:25.460085 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ce8a7-b2f8-402c-8124-3a978cb9a554 service nova] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:49:25.460366 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ce8a7-b2f8-402c-8124-3a978cb9a554 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:49:25.462927 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1d8ce8a7-b2f8-402c-8124-3a978cb9a554 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:49:25.474820 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ce8a7-b2f8-402c-8124-3a978cb9a554 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:49:25.476508 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ce8a7-b2f8-402c-8124-3a978cb9a554 service nova] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:49:25.486904 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1454/2932] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:49:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:49:35.481959 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=34 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:50:32.822331 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e99b80-cfb1-40df-99cc-c1e086435e16 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:32.823922 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c8e99b80-cfb1-40df-99cc-c1e086435e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:32.907627 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e99b80-cfb1-40df-99cc-c1e086435e16 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:32.913681 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed225b61-79ee-4468-b8e0-f9d3f79ebd40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6281dcf2927a4ed2870125df8801be78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:32.914483 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed225b61-79ee-4468-b8e0-f9d3f79ebd40 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:32.915152 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed225b61-79ee-4468-b8e0-f9d3f79ebd40 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:32.915152 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed225b61-79ee-4468-b8e0-f9d3f79ebd40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:32.917110 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ed225b61-79ee-4468-b8e0-f9d3f79ebd40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:32.930325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed225b61-79ee-4468-b8e0-f9d3f79ebd40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:32.931843 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed225b61-79ee-4468-b8e0-f9d3f79ebd40 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:32.944084 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1479/2933] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:50:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:50:42.939268 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=32 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:51:11.037053 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ce8a7-b2f8-402c-8124-3a978cb9a554 service nova] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:11.039038 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-1d8ce8a7-b2f8-402c-8124-3a978cb9a554 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:11.119778 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ce8a7-b2f8-402c-8124-3a978cb9a554 service nova] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:11.125673 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7c04e-e903-4319-bdac-dac55f11b5e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:11.126466 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7c04e-e903-4319-bdac-dac55f11b5e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:11.126746 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7c04e-e903-4319-bdac-dac55f11b5e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:11.127012 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7c04e-e903-4319-bdac-dac55f11b5e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:11.129296 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-97c7c04e-e903-4319-bdac-dac55f11b5e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:11.151412 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7c04e-e903-4319-bdac-dac55f11b5e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:11.153710 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7c04e-e903-4319-bdac-dac55f11b5e8 service neutron] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:11.167810 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1455/2934] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:51:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:51:16.806734 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed225b61-79ee-4468-b8e0-f9d3f79ebd40 service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:16.808484 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ed225b61-79ee-4468-b8e0-f9d3f79ebd40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:16.825220 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed225b61-79ee-4468-b8e0-f9d3f79ebd40 service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:16.830456 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093dc3e-a891-4e26-a920-bd22d50355b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0718a95189434a5ea901c40bc868bea4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:16.831091 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093dc3e-a891-4e26-a920-bd22d50355b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:16.831368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093dc3e-a891-4e26-a920-bd22d50355b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:16.831692 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093dc3e-a891-4e26-a920-bd22d50355b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:16.835103 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a093dc3e-a891-4e26-a920-bd22d50355b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:16.914867 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093dc3e-a891-4e26-a920-bd22d50355b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:16.916368 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093dc3e-a891-4e26-a920-bd22d50355b6 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:16.927418 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1480/2935] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:51:20.941842 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7c04e-e903-4319-bdac-dac55f11b5e8 service neutron] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:20.943782 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-97c7c04e-e903-4319-bdac-dac55f11b5e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:20.962712 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7c04e-e903-4319-bdac-dac55f11b5e8 service neutron] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:20.969205 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f988c7b-637f-4957-82c5-2fbd8cfc3a61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6281dcf2927a4ed2870125df8801be78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:20.969845 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f988c7b-637f-4957-82c5-2fbd8cfc3a61 service placement] REQUEST_METHOD: `GET` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:20.970145 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f988c7b-637f-4957-82c5-2fbd8cfc3a61 service placement] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:20.970392 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f988c7b-637f-4957-82c5-2fbd8cfc3a61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:20.972564 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-7f988c7b-637f-4957-82c5-2fbd8cfc3a61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:20.986351 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f988c7b-637f-4957-82c5-2fbd8cfc3a61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:20.988119 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f988c7b-637f-4957-82c5-2fbd8cfc3a61 service placement] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:21.001044 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1456/2936] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:51:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:51:24.420088 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093dc3e-a891-4e26-a920-bd22d50355b6 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:24.421838 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a093dc3e-a891-4e26-a920-bd22d50355b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:24.437247 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093dc3e-a891-4e26-a920-bd22d50355b6 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:24.442822 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48d56e-c00c-4446-b674-d522bba36f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:24.443543 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48d56e-c00c-4446-b674-d522bba36f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:24.443844 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48d56e-c00c-4446-b674-d522bba36f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:24.444259 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48d56e-c00c-4446-b674-d522bba36f26 admin admin] PATH_INFO: `/v3/users/ed4710f614cc431683a9c2a4f3d9d7b6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:24.445392 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48d56e-c00c-4446-b674-d522bba36f26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed4710f614cc431683a9c2a4f3d9d7b6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:24.448341 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48d56e-c00c-4446-b674-d522bba36f26 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:24.500955 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf48d56e-c00c-4446-b674-d522bba36f26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed4710f614cc431683a9c2a4f3d9d7b6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:24.509325 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf48d56e-c00c-4446-b674-d522bba36f26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed4710f614cc431683a9c2a4f3d9d7b6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:24.509932 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf48d56e-c00c-4446-b674-d522bba36f26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed4710f614cc431683a9c2a4f3d9d7b6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:24.522161 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf48d56e-c00c-4446-b674-d522bba36f26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed4710f614cc431683a9c2a4f3d9d7b6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:24.529942 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf48d56e-c00c-4446-b674-d522bba36f26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed4710f614cc431683a9c2a4f3d9d7b6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:24.537059 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1481/2937] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:51:24 2022] DELETE /identity/v3/users/ed4710f614cc431683a9c2a4f3d9d7b6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:51:24.728616 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f988c7b-637f-4957-82c5-2fbd8cfc3a61 service placement] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:24.741166 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f988c7b-637f-4957-82c5-2fbd8cfc3a61 service placement] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:24.813326 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:24.813968 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:24.814237 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:24.814503 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] PATH_INFO: `/v3/projects/ec4d86f1d7bc4cc9a39ff55d4dc6a801` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:24.815642 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec4d86f1d7bc4cc9a39ff55d4dc6a801)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:24.817652 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:24.869484 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec4d86f1d7bc4cc9a39ff55d4dc6a801', 'request_id': 'req-0c49a43b-7747-4060-a967-b3b2e5bc7b91'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:24.877578 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec4d86f1d7bc4cc9a39ff55d4dc6a801', 'request_id': 'req-0c49a43b-7747-4060-a967-b3b2e5bc7b91'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:24.883649 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec4d86f1d7bc4cc9a39ff55d4dc6a801', 'request_id': 'req-0c49a43b-7747-4060-a967-b3b2e5bc7b91'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:24.895870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] The token cache is being invalidate because project ec4d86f1d7bc4cc9a39ff55d4dc6a801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:51:24.895870 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:24.896462 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:24.897993 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1457/2938] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:51:24 2022] DELETE /identity/v3/projects/ec4d86f1d7bc4cc9a39ff55d4dc6a801 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:51:24.902337 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48d56e-c00c-4446-b674-d522bba36f26 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:24.904208 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bf48d56e-c00c-4446-b674-d522bba36f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:24.920047 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48d56e-c00c-4446-b674-d522bba36f26 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:24.984502 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb7f60-e4c8-40e3-b05e-40990f36bd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:24.985147 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7f60-e4c8-40e3-b05e-40990f36bd7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:24.985443 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7f60-e4c8-40e3-b05e-40990f36bd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:24.985657 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7f60-e4c8-40e3-b05e-40990f36bd7d admin admin] PATH_INFO: `/v3/users/9d52bc941fb2491091249353f1c677e5` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:24.986888 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb7f60-e4c8-40e3-b05e-40990f36bd7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d52bc941fb2491091249353f1c677e5)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:24.988842 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb7f60-e4c8-40e3-b05e-40990f36bd7d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:24.994630 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:51:25.042525 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-47cb7f60-e4c8-40e3-b05e-40990f36bd7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d52bc941fb2491091249353f1c677e5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:25.051345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-47cb7f60-e4c8-40e3-b05e-40990f36bd7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d52bc941fb2491091249353f1c677e5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:25.051958 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-47cb7f60-e4c8-40e3-b05e-40990f36bd7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d52bc941fb2491091249353f1c677e5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:25.064070 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-47cb7f60-e4c8-40e3-b05e-40990f36bd7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d52bc941fb2491091249353f1c677e5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:25.072278 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-47cb7f60-e4c8-40e3-b05e-40990f36bd7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d52bc941fb2491091249353f1c677e5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:25.078916 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1482/2939] 10.222.0.62 () {58 vars in 1224 bytes} [Thu Sep 1 16:51:24 2022] DELETE /identity/v3/users/9d52bc941fb2491091249353f1c677e5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:51:25.260407 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] Authenticating user token {{(pid=81523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:25.261950 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.fernet_utils [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:25.275447 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49a43b-7747-4060-a967-b3b2e5bc7b91 admin admin] Validating token access rules against request {{(pid=81523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:25.295385 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:51:25.338759 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca8812-22af-4863-9909-edaef9334584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b69c02f3fd4ddc81686f03143f0651', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '026b4ed9c3cf48a48d86818cba23c13c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:25.339413 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca8812-22af-4863-9909-edaef9334584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:25.339692 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca8812-22af-4863-9909-edaef9334584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:25.339991 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca8812-22af-4863-9909-edaef9334584 admin admin] PATH_INFO: `/v3/projects/0fcaa980a74a40ad940f0a53e18a77e5` {{(pid=81523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:25.341315 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca8812-22af-4863-9909-edaef9334584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fcaa980a74a40ad940f0a53e18a77e5)` {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:25.343352 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca8812-22af-4863-9909-edaef9334584 admin admin] RBAC: Authorization granted {{(pid=81523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:25.397015 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6cca8812-22af-4863-9909-edaef9334584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fcaa980a74a40ad940f0a53e18a77e5', 'request_id': 'req-6cca8812-22af-4863-9909-edaef9334584'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:25.406088 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6cca8812-22af-4863-9909-edaef9334584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fcaa980a74a40ad940f0a53e18a77e5', 'request_id': 'req-6cca8812-22af-4863-9909-edaef9334584'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:25.412595 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6cca8812-22af-4863-9909-edaef9334584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fcaa980a74a40ad940f0a53e18a77e5', 'request_id': 'req-6cca8812-22af-4863-9909-edaef9334584'} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:25.424705 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6cca8812-22af-4863-9909-edaef9334584 admin admin] The token cache is being invalidate because project 0fcaa980a74a40ad940f0a53e18a77e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:51:25.425024 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6cca8812-22af-4863-9909-edaef9334584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:25.426002 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG keystone.notifications [None req-6cca8812-22af-4863-9909-edaef9334584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:25.427606 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: [pid: 81523|app: 0|req: 1458/2940] 10.222.0.62 () {58 vars in 1233 bytes} [Thu Sep 1 16:51:25 2022] DELETE /identity/v3/projects/0fcaa980a74a40ad940f0a53e18a77e5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:51:35.075024 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:51:35.421398 dv-cinder-852210-9-iscsi devstack@keystone.service[81523]: DEBUG dbcounter [-] [81523] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:55:38.784345 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb7f60-e4c8-40e3-b05e-40990f36bd7d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:55:38.786430 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-47cb7f60-e4c8-40e3-b05e-40990f36bd7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:55:38.979472 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb7f60-e4c8-40e3-b05e-40990f36bd7d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:55:39.050699 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45871e6-29c5-4936-8acc-d1e6d4a16bef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6281dcf2927a4ed2870125df8801be78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425048ffd13448cd8fe815333a9b798c', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:55:39.051244 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45871e6-29c5-4936-8acc-d1e6d4a16bef service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:55:39.051514 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45871e6-29c5-4936-8acc-d1e6d4a16bef service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:55:39.051776 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45871e6-29c5-4936-8acc-d1e6d4a16bef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:55:39.054461 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d45871e6-29c5-4936-8acc-d1e6d4a16bef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:55:39.066374 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45871e6-29c5-4936-8acc-d1e6d4a16bef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:55:39.067984 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45871e6-29c5-4936-8acc-d1e6d4a16bef service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:55:39.078854 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1483/2941] 10.222.0.62 () {60 vars in 1365 bytes} [Thu Sep 1 16:55:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:55:49.074340 dv-cinder-852210-9-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}